Triage code security findings and open draft fix PRs
Last updated
The problem
Static analysis, secret scanners, and dependency scanners flood repositories with alerts, many of which are false positives in test fixtures or unreachable code paths. Engineers start ignoring the queue, real vulnerabilities sit unpatched, and nobody has time to trace every warning through the codebase and write the fix.
How the agent works
When a code scanner or commit hook reports a finding (such as a SQL injection sink, missing input validation, an exposed secret pattern, or a vulnerable package usage), the agent reads the alert alongside the surrounding files and call graph in a sandboxed checkout. It checks whether untrusted input actually reaches the flagged line, classifies the alert as likely exploitable, defence-in-depth, or false positive (for example, a mock token in a unit test), and posts its reasoning with file and line references. For clear code-level fixes, it opens a draft pull request and runs the test suite so an engineer can review a working patch.
Setup steps
- Connect the agent to your SAST, secret-scanning, or dependency-alert webhook in a read-only sandbox environment
- Have the agent trace callers and data flow around each alert to see whether user-controlled input reaches the flagged code path
- Require a structured triage comment: severity assessment, reachability evidence with file and line links, and whether the file is production code or a test fixture
- For leaked secrets that look real, have the agent alert the security owner immediately to rotate the credential rather than just deleting the line in git
- For straightforward code flaws (such as unparameterized queries, missing escaping, or unsafe deserialization), let it open a draft PR with a regression test
- Have a developer or security engineer review every dismissal and every fix PR before anything is closed or merged
Tools that can do this
- Claude Code
- GitHub Copilot coding agent
- Cursor
- Devin
- Codex
- GitHub Actions
Not ranked. Tools marked Featured pay for a highlighted listing; that never decides which tools we include. Check each vendor's current features and pricing.
Make one of these tools? Feature your tool on every playbook where it is already listed.
Human in the loop
A human engineer or security owner approves every alert dismissal and reviews and merges every fix pull request. Real credential leaks always trigger human rotation and incident checks.
Risks and guardrails
An agent can mistakenly label a real vulnerability as a false positive if dynamic routing or reflection hides the call path, so never let it auto-dismiss alerts without human sign-off. Never paste live leaked secrets into third-party prompts or public PR comments. Run the agent in an isolated sandbox without production credentials.
ROI notes
Measure median time to triage new security alerts, the backlog of unreviewed findings, and the share of draft fix PRs that engineers accept with minor or no edits.
Also useful: get a first-pass review on every pull request; upgrade dependencies and fix breaking changes; hand scoped issues to a coding agent.
Related playbooks
Get a first-pass review on every pull request
Reviewers are a bottleneck and miss routine issues.
Hand well-scoped issues to a coding agent
Small bugs and chores sit in the backlog because nobody has time.
Write missing unit tests for legacy code
Untested legacy code makes every change risky.
All AI agent examples, by function · More Software Engineering playbooks