Agentics

Appearance

TL;DR — Primitive

A Review Report is a risk-tiered audit document produced after a builder agent finishes — not by the builder, but by an independent reviewer reading the plan and the diff. Its machine-readable verdict (PASS or FAIL) is the only gate between code and PR.

Primitive

Review Report ​

The Artifact

What a Review Report looks like on disk ​

app_review/review_20260417T143012.md

# Review ReportTimestamp: 2026-04-17T14:30:12ZPlan Reference: app_plan/plan_20260417T091500.mdDiff Reviewed: feat/add-payment-webhook (47 files changed)Verdict: FAIL────────────────────────────────────────────## BLOCKER- [ ] Webhook signature validation skipped on retry path File: src/payments/webhook.ts Line: 84 Plan claim: "all payloads validated before processing" Actual: validateSignature() not called in retryEvent()────────────────────────────────────────────## HIGH- [ ] No idempotency key check before DB write File: src/payments/processor.ts Line: 212 Risk: duplicate charges on network retry────────────────────────────────────────────## MEDIUM- [x] Error logging omits payment ID in catch block File: src/payments/webhook.ts Line: 61────────────────────────────────────────────## LOW- [x] Unused import in webhook.ts:3────────────────────────────────────────────Summary: 1 BLOCKER1 HIGH1 MEDIUM1 LOW

The file lives at app_review/review_<timestamp>.md. One BLOCKER is enough to set the verdict to FAIL. Checked boxes are acknowledged issues — unchecked are open.

The Four Tiers

Every issue is classified before it is reported ​

Blocker

Must fix before merge

Correctness failures, security gaps, or direct contradictions of the Plan. A single BLOCKER forces FAIL and halts the PR pipeline unconditionally.

High

Strong recommendation

Significant risk — data loss, race conditions, missing idempotency. Does not force FAIL alone, but stacking HIGHs should trigger escalation.

Medium

Fix or document

Observable quality gap — missing log context, incomplete error handling. Accepted only when a comment in code explicitly acknowledges the trade-off.

Low

Optional polish

Style, dead imports, minor inconsistencies. Noted for completeness. Never influences the PASS/FAIL verdict. Fix in a cleanup pass or ignore.

Tiers are assigned by the reviewer, not by the builder. Classification is a judgment call against the plan, not against convention.

The Decision Gate

Verdict is a machine-readable branch point ​

FAIL

  • ▲PR creation is blocked. The pipeline does not open a pull request. No human review is requested.
  • ▲Fix agent receives the report. The structured issue list — file, line, tier — is passed directly as context. No re-summarisation.
  • ▲Re-review is required. After fixes are applied, the review command runs again. A new report is produced from the updated diff.
  • ▲Loop terminates only on PASS. Fix + re-review cycles repeat until no BLOCKERs remain.

PASS

  • ▶PR creation proceeds. The pipeline opens the pull request automatically. Human review begins.
  • ▶MEDIUM and LOW issues are attached. They appear in the PR body as advisory notes for the human reviewer — not as blockers.
  • ▶Report is linked, not paraphrased. The PR references the report file path so reviewers can audit the full classification.
  • ▶Quality gate is machine-enforced. No human decided to skip review. The gate was cleared on evidence.

The verdict string PASS or FAIL is parsed by the orchestrator — not inferred from prose. Ambiguous output defaults to FAIL.

What the Reviewer Reads

The report is the output of an independent check — not a self-assessment ​

Builder agent says

"Task complete. All requirements implemented per plan. Tests pass."

  • Self-reported — no adversarial check
  • Tests passing != plan claims satisfied
  • Blind to its own omissions by construction
  • No structured issue list produced

Reviewer reads

"Plan claimed X. Diff shows Y. Checking claim by claim."

  • Reads the original plan spec, not the builder summary
  • Reads the raw diff, not the builder's description of it
  • Checks each plan claim as a falsifiable assertion
  • Produces a tiered issue list regardless of builder confidence

The reviewer is a separate agent invocation with no access to the builder's internal state. Trust is replaced by evidence.

Why It Matters ​

Quality gates become machine-enforceable rather than trust-based.

Fix agents receive a structured, prioritized issue list — not vague feedback. Code with blocking issues cannot advance to PR. The pipeline enforces the standard automatically, every time, without human intervention in the loop.