AI review first, and filtered

Why

An unfiltered automated reviewer can slow a team down. In one industrial study, the average time to close a pull request rose from five hours and 52 minutes to eight hours and 20 minutes after an LLM-based reviewer was introduced, even though developers resolved 73.8 percent of its comments ^[10]. HubSpot added a second “judge” agent whose only job is to filter the first reviewer’s comments for succinctness, accuracy and actionability before anything is posted, and calls it the single most important factor in the reviewer’s effectiveness ^[9].

Uber’s reviewer has filtering, validation and deduplication stages behind it. It covers more than 90 percent of the roughly 65,000 changes landed each week, and engineers rate 75 percent of its comments useful ^[8]. In this playbook, the automated first pass is there to handle the extra pull requests that rule 6 creates, so the human reviewer’s queue stays manageable.

How

The workflow runs Claude Code as a GitHub Action on every non-draft pull request. The prompt does the filtering. It reads the instruction file, the constitution, the review policy, the deliberate deviations in section 7 of the system design, and the spec the pull request names, then reports only these:

1. Important: the change does not do what the PR's Verification
   section claims, or the evidence is missing.
2. Important: a cited requirement has no test that would fail
   without this change.
3. Important: a contract changed without both guard tests changing.
4. Important: anything named after a test case or a fixture value.
5. Important: a constitution non-negotiable is broken, or a
   deliberate deviation in the design is "fixed".
6. Important: secrets, unhandled null paths, swallowed exceptions,
   a schema change without a migration.
7. Nit: at most three, only where the fix is one line.
8. Pre-existing: an Important-level problem in lines this PR did
   not change. Not blocking. At most one.

Do not comment on style the linter covers. Do not summarize the diff.
If there is nothing Important, post one top-level comment saying so.

Findings are posted inline, tagged Important, Nit or Pre-existing, the same tags Anthropic’s managed review uses. Only Important findings need an answer.

One line in the workflow is easy to miss. When a workflow gives the action a prompt, the action grants Claude no tools until the workflow names them. The kit’s claude_args line passes --allowedTools with git diff, git log, the inline comment tool and gh pr comment. Without it the reviewer can neither read the change nor post, and the findings only reach the run log. Without gh pr comment it cannot post the top-level comment that says nothing Important was found.

Four rules for tuning it. Start strict: Important findings, at most three nits, at most one pre-existing finding, and nothing else. Watch the time to close a pull request. If it rises, the reviewer is probably too chatty. The author answers every Important finding with a fix or a one-line reason, and the dismissed ones are the data for tuning. Tune it in the repository: Anthropic’s managed review reads a REVIEW.md, Cursor’s Bugbot reads .cursor/BUGBOT.md, Codex reads a section of AGENTS.md.

The writer and the reviewer are different sessions. When one agent writes and reviews code in the same session, it reviews its own work with the same context it wrote it in. The kit’s reviewer agent runs before the pull request is opened, with fresh context and read-only access. It checks the same list as the workflow, plus whether files outside the task’s Touches list were changed and whether the documents the change made wrong were updated. Findings are tagged Important or Nit, with no praise and no summary of the diff.