AI review first, and filtered

Why

An unfiltered automated reviewer makes things worse. In one industrial study, the average time to close a pull request rose from five hours and 52 minutes to eight hours and 20 minutes after an LLM-based reviewer was introduced, even though developers resolved 73.8 percent of its comments ^[10]. HubSpot added a second “judge” agent whose only job is to filter the first reviewer’s comments for succinctness, accuracy and actionability before anything is posted, and calls it the single most important factor in the reviewer’s effectiveness ^[9].

Filtered, it works. Uber’s reviewer covers more than 90 percent of the roughly 65,000 changes landed each week, and engineers rate 75 percent of its comments useful, because the team built filtering, validation and deduplication stages behind it ^[8]. The first pass absorbs the volume that rule 6 creates, so the human queue stays flat.

How

The workflow runs Claude Code as a GitHub Action on every non-draft pull request. The prompt is the filter. It reads the instruction file, the constitution, the review policy and the spec the pull request names, then reports only these:

1. Important: the change does not do what the PR's Verification
   section claims, or the evidence is missing.
2. Important: a cited requirement has no test that would fail
   without this change.
3. Important: a contract changed without both guard tests changing.
4. Important: anything named after a test case or a fixture value.
5. Important: a constitution non-negotiable is broken, or a
   deliberate deviation in the design is "fixed".
6. Important: secrets, unhandled null paths, swallowed exceptions,
   a schema change without a migration.
7. Nit: at most three, only where the fix is one line.

Do not comment on style the linter covers. Do not summarize the diff.

Four rules keep it trusted. Start with Important findings only and at most three small ones. Watch the time to close a pull request, and if it rises, the reviewer is too chatty, not the developers too slow. The author answers every Important finding with a fix or a one-line reason, and the dismissed ones are the data for tuning. Tune it in the repository: Anthropic’s managed review reads a REVIEW.md, Cursor’s Bugbot reads .cursor/BUGBOT.md, Codex reads a section of AGENTS.md.

The writer and the reviewer are different sessions. When one agent both writes and reviews code in the same session, it grades its own homework. The kit’s reviewer agent runs before the pull request is opened, with fresh context and read-only access. It checks the same list as the workflow, plus whether files outside the task’s Touches list were changed and whether the documents the change made wrong were updated. Findings are tagged Important or Nit, with no praise and no summary of the diff.