
What happened
A PM built an AI-run workflow across 3 personal repos (podcast, personal site, monetization tool) using a Can / May / Act permission model and a machine-readable merge gate keyed on Greptile's Confidence Score.
Why it matters
The gate works as a feedback signal, not just a checkpoint — because a PR is not merged until the condition is met, that condition itself pushes the AI to keep fixing until it passes.
Summaries like this, in your inbox every morning.
The experiment did not start by handing an agent full authority. Permissions were layered into Can (the range of tools, commands, and domains the agent can physically reach, enforced through declarative allowlists), May (an approval step at execution time, such as GitHub author-association gates and AI classifiers that block dangerous commands), and Act (artifacts only gain real effect after passing a gate, e.g. audio uploads stopping at Draft or CLI writes defaulting to dry-run).
The most consequential part was making merge conditions machine-readable. On yukamiya.me, a PR needed three conditions together: a Greptile Confidence Score of 5/5, all required checks green, and a machine-readable AUTO_REVIEW_STATUS: pass marker. On hub.momit.fm, a GitHub Actions workflow named greptile-gate parsed Confidence Score: X/5 from a comment and ran gh pr merge only on a perfect score. Because the gate refused to merge until the condition was met, an AI reviewer's score on a PR adding the autonomous review skill itself moved from 0/5 to 3/5 to 5/5, and the final iteration was auto-merged without any human-written line.
The failures shaped the guardrails as much as the successes. A weekly cron job on macOS failed silently for 18 days under Operation not permitted due to TCC privacy limits, and because manual runs worked the problem went unnoticed — monitoring automation must itself be automated. Greptile sometimes updated a summary comment in place instead of creating a new review object, which timed out a gate waiting only for review objects, so detection was split into two channels. An external AI reviewer caught that the review skill itself hard-coded an absolute path it had flagged elsewhere, showing that self-verification has structural blind spots. And because branch names in a public repo are attacker-controlled input, branch-name format checks, required quoting, and automatic permission downgrade for fork PRs were added.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.