Edison SinaniAI implementation architectBook a call

AI Team Playbook / Rule 9 of 12

Human review by risk tier

Every task carries a risk tier, set by its author and checked by its reviewer, and the tier decides who has to review it and how closely.

September 16, 2026 · About 2 minutes

Contents

The playbook

  1. Overview

Before code

  1. 1Write the constitution first
  2. 2One instruction file, under 200 lines
  3. 3Prose is advice, hooks are enforcement
  4. 4Three gates before any code
  5. 5A walkable demo before any screen

Building

  1. 6One task, one worktree, one pull request
  2. 7Every contract has a file and two tests

Reviewing

  1. 8AI review first, and filtered
  2. 9Human review by risk tier
  3. 10Done means the evidence is in the pull request

Over time

  1. 11Measure five numbers from day one
  2. 12Protect the people who are still learning

Appendix

  1. AInstall checklist
  2. BOther tools, limits and sources
  3. CTightening the process
  4. DRun the agent with no entitlements

Get the kit

Who owns it
The author sets the tier. The reviewer checks it. The lead reviews every High task, with one other person.
In the kit
docs/review-policy.md.github/PULL_REQUEST_TEMPLATE.mddocs/governance.md
Enforced by
The pr-template check in CI fails a pull request with no tier chosen. Branch protection requires an approving review.

Why

Zalando classifies every pull request by rollout risk, auto-approves the 33 percent it rates low, and measured a 20 to 40 percent cut in pull request lead time [11]. Anthropic tiers its own codebase by risk, automates review according to the tier, logs every automated approval with the signals it used, and has humans review a risk-weighted sample [12]. Adding reviewers does not scale. Sorting the queue does.

How

Tier What is in it Human review Merges when
Low Additive and reversible. No contract, no authentication, no money, no data migration, no production configuration. Any team member, and it may be light CI green, one approval
Standard Everything not Low or High. One team member who did not write it, reading for conformance to the design and for the verification evidence CI green, one approval
High Authentication, permissions, billing, migrations that change or delete data, anything under contracts/, production infrastructure and secrets, integrations that send data out. The lead plus one other. The lead checks the evidence personally. CI green, both approvals

When in doubt, go up a tier. A typo in configuration metadata caused an incident at Zalando, and their risk tool now rates that kind of change high [11].

What the human looks at. Not a second linter. Five things, in order.

  1. Does the change do what the task’s verification line says, and is the evidence in the pull request?
  2. Does it follow the feature design and the constitution’s non-negotiables?
  3. Is anything in it named after a test case, a fixture value or one specific input?
  4. Did the automated first pass flag anything the author dismissed, and was the reason good?
  5. Would a developer joining next month understand this change from the pull request description alone?

If the reviewer cannot answer the first question from the pull request itself, it goes back. This is the single most useful habit for a team using agents.

Automatic approval for Low.

Not from day one. After one full milestone, when the weekly numbers from rule 11 show the automated first pass catching what humans catch on Low changes, the lead may switch Low to “automated review plus green CI merges without a human”. Keep a sample for a human to look at afterwards, as Anthropic does [12], and write the switch-on date and the evidence in the governance file.

High tier work is reviewed in pairs, the lead and one other person, rotating, so everyone has seen the sharp edges.

References

  1. 11Zalando Engineering, "Agentic Engineering at Zalando: a snapshot", August 2026. engineering.zalando.com/posts/2026/08/agentic-engineering-at-zalando-a-snapshot.html
  2. 12Anthropic, "How Anthropic secures its AI-native software development lifecycle", July 2026. claude.com/blog/how-anthropic-secures-its-ai-native-software-development-lifecycle