More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
I spent months turning our security‐patch workflow into a mix of code and AI agents. Every time Dependabot flags a vulnerability, a webhook launches our internal framework. A script first weeds out low‐priority or resolved issues, then bundles each remaining alert by repository. Those bundles go to an LLM that uses our “ownership” skill plus GitHub’s MCP logic—checking CODEOWNERS files, recent commits, and so on—to pick the right teams or individuals. A second agent then formats and posts a clean Slack notification.
At first I tried doing everything inside the agent prompt: filter by critical severity, assign owners, format messages. It broke down as soon as I pushed it harder—high‐ and medium‐severity alerts slipped through. Instead of endless prompt tinkering or building a hard eval to block wrong messages, I shifted most of the workflow back into plain code. Now software handles the deterministic steps—filtering, batching, data extraction—and only cedes control when human-style judgment is needed, like matching people to code areas.
That hybrid approach works flawlessly. We rolled out real-time critical‐issue alerts without worrying the bot will spam non-critical items. Next up is a weekly summary ping for open critical issues and eventually auto-generating the fixes themselves, leaving only a quick human review before deployment. Over time I’ve found this scaffold—prototype in pure agent, then refactor toward code with just a few agent calls—delivers faster, cheaper, more reliable automation.
Questions about this article
No questions yet.