Skip to content

feat: add py-agent for Python workflow checks#1

Merged
codesoda merged 6 commits intomainfrom
feat/py-agent
Mar 8, 2026
Merged

feat: add py-agent for Python workflow checks#1
codesoda merged 6 commits intomainfrom
feat/py-agent

Conversation

@codesoda
Copy link
Owner

@codesoda codesoda commented Mar 7, 2026

Summary

  • Adds py-agent — lean Python workflow runner (format, lint, typecheck, test)
  • Auto-detects runner (uv, poetry, plain) and tools (ruff, black, flake8, mypy, pyright, pytest)
  • CI-aware: format checks in CI, auto-fixes locally; full output in CI
  • Supports --fail-fast, Fix: hints, and all shared env knobs

Test plan

  • Smoke test clean fixture (all steps PASS/SKIP)
  • Smoke test issues fixture (test FAIL with diagnostics)
  • --help output verified
  • Test with ruff/black/mypy installed in a real project
  • Test uv and poetry runner detection
  • Run tests/run-scenarios.sh py-agent

codesoda added 3 commits March 7, 2026 15:11
- Add py-agent skill and script (format, lint, typecheck, test)
- Auto-detect runner (uv, poetry, plain) and python binary (python3/python)
- Auto-detect tools: ruff/black (format), ruff/flake8 (lint),
  mypy/pyright (typecheck), pytest/unittest (test)
- CI-aware: format checks in CI, auto-fixes locally; MAX_LINES unlimited in CI
- Support --fail-fast flag (passes -x to pytest)
- Fix hints after each failure with auto-fix suggestions where available
- Add clean and issues scenario test fixtures
- Update README, install.sh with py-agent entry
In fix mode, run --check first to detect which files need formatting,
then apply the fix and list the changed files in the output.
Ported from ai-barometer's cargo-agent.
@codesoda codesoda marked this pull request as ready for review March 8, 2026 07:27
Copy link

@ai-barometer-staging ai-barometer-staging bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cadence Session Review

No AI session data was found for this PR's commits.

Sign up to Cadence to have your AI sessions reviewed

Copy link

@cadence cadence bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cadence Session Review

Score B+ Solid execution with minor scope gaps and one ordering bug
Models Claude (opus-4.6)
Sessions 5
Phases 60% implementation, 20% cross-cutting refactors, 15% debugging, 5% documentation

Model delivered py-agent plus cross-cutting improvements (fail-fast, Fix: hints, CI-aware MAX_LINES) to all existing agents in a single sweep. Followed output contract conventions well, including scenario fixtures.

  • py-agent script is well-structured, mirrors cargo/npm/terra patterns faithfully
  • Cross-cutting refactors (fail-fast, Fix: hints, CI MAX_LINES) applied consistently across all agents
  • CI workflow not updated to include py-agent scenarios — scope gap from session ordering
  • --help broken outside Python project dirs due to check ordering
  • Session included good iterative debugging (inverted logic fix, python3 fallback)

Recommendations

Prompting — Enumerate all cross-cutting files in the prompt

When adding a new agent that should follow existing patterns (including CI), explicitly mention all artifacts that need updating. The model applied changes to docs and install.sh but missed ci.yml because the CI workflow was created in a separate session.

Before
The CI workflow was created in an earlier session, and the py-agent session never referenced updating it.

Reframe
"Add py-agent with full workflow. Also update ci.yml to add a py-agent job matching the existing patterns."

Tip
Keep a checklist of cross-cutting files (ci.yml, install.sh, README.md) and name them in the prompt.


Agent instructions — Add help-accessibility rule to agent conventions

The model placed the Python project existence check before the help case, breaking --help outside a project. Adding an explicit rule in AGENTS.md about help accessibility would prevent this pattern across all agents.

Reframe
Add to AGENTS.md: - The help/usage command must work without any project context (no project detection before help).


Prompting — Request edge-case testing for CLI flags

When verifying CLI behavior, prompt the model to test edge cases like running from outside a valid project directory. The model only tested happy paths from within fixture dirs.

Before
The model only tested --help from within the fixture directory.

Reframe
After running --help, also run: cd /tmp && py-agent.sh --help to verify help works outside a project directory.

Tip
A good pattern: 'Test the command from both inside and outside a valid project directory.'

codesoda added 3 commits March 8, 2026 19:07
- Add py-agent CI job with baseline and with-ruff matrix variants
- Add workflow-level flock (Linux) / Perl fallback (macOS) to prevent
  concurrent py-agent runs
- Add CHANGED_FILES support: scopes format and lint to changed .py
  files; typecheck and test run project-wide (need full context)
- Add CHANGED_FILES to SKILL.md allowed-tools and env knobs table
- Move help/--help before project-existence checks (py-agent + convention)
- Switch have_tool() to use `which` via runner instead of --version
- Make unittest fallback conditional on python being available
- Add SKIP result when no test runner is found
- Update definition-of-done with help-accessibility requirement
@codesoda codesoda merged commit 7b938a0 into main Mar 8, 2026
8 checks passed
@codesoda codesoda deleted the feat/py-agent branch March 8, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant