--- framework: Agentic Engineering Maturity — Factory Edition version: 2026.06.0 kind: concepts-primer --- # The 2026 Concepts Behind the Checklists A short field glossary. Each entry is the minimum you need to read the checklists fluently, plus where the idea comes from. The assessment assumes these as background; the companion guides reference them by name. > These ideas crystallized across 2026 in writing by **Addy Osmani** and others. Primary sources are linked at the end of each entry. The data points are quoted because they are the reason these practices stopped being optional. --- ## 1. The Factory Model Software engineering moved up an abstraction level — again. The job is shifting from *writing code* to *building and running the system that writes code*: fleets of agents, each with a task, a toolbelt, context, and a feedback loop, with you doing quality control. Take the factory analogy seriously and it tells you where to invest: precise inputs (specs), quality control (verification), reliable environments (sandboxes, fast provisioning), and process documentation (context the agents can read). The corollary that matters most: **your spec is the leverage.** When you orchestrate dozens of parallel agents, the gap between mediocre and exceptional output is almost entirely the quality of your specification, because ambiguity multiplies across every run. *Source: Addy Osmani, ["The Factory Model: How Coding Agents Changed Software Engineering"](https://addyosmani.com/blog/factory-model/) (Feb 2026).* --- ## 2. Loop Engineering > "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents." — Peter Steinberger A *loop* is a system that finds work, hands it to an agent, checks it, records what's done, and decides the next thing — so the system prompts the agent instead of you. Five building blocks plus memory: 1. **Automations** — scheduled discovery and triage (the heartbeat). 2. **Worktrees** — isolation so parallel agents don't collide. 3. **Skills** — project knowledge written down so the agent doesn't re-guess it every run. 4. **Plugins / connectors (MCP)** — the loop touches your real tools (issues, DB, CI, chat). 5. **Sub-agents** — one has the idea, a *different* one checks it. 6. **State / memory** — a file or board outside the conversation that remembers what's done and what's next. The skill that replaces prompt engineering is *loop design* — and it's harder, not easier, because the loop runs while you aren't watching. *Source: Addy Osmani, ["Loop Engineering"](https://addyosmani.com/blog/loop-engineering/) (June 2026).* --- ## 3. Agent Harness Engineering **Agent = Model + Harness.** The harness is everything that isn't the model: system prompts, `AGENTS.md`, tools, MCP servers, sandboxes, hooks, sub-agent orchestration, context policies, and recovery paths. *If you're not the model, you're the harness.* Key principles: - **A decent model with a great harness beats a great model with a bad harness.** (Demonstrated: the same model jumped from Top 30 to Top 5 on a benchmark by changing only the harness.) - **The ratchet** — every time an agent makes a mistake, you add a constraint (a line in `AGENTS.md`, a hook, a reviewer check) so it can never make that mistake again. *Every line in a good `AGENTS.md` traces to a specific failure.* - **Work backwards from behavior** — if you can't name the behavior a harness component delivers, it shouldn't be there. - **Hooks: success is silent, failure is verbose.** A passing check says nothing; a failing one injects the error back into the loop so the agent self-corrects. - **Battling context rot** — compaction, tool-output offloading, and progressive disclosure keep the model sharp as the window fills. - **Planner / generator / evaluator splits** outperform self-evaluation, because agents skew positive grading their own work. *Source: Addy Osmani, ["Agent Harness Engineering"](https://addyosmani.com/blog/agent-harness-engineering/) (Apr 2026); Anthropic, ["Building agents for long-running work"](https://www.anthropic.com/engineering/harness-design-long-running-apps).* --- ## 4. Agentic Code Review & the Verification Bottleneck Writing got cheap; understanding stayed expensive. So the constraint moved downstream to the one step that didn't speed up: **a person being confident the change is right.** The 2026 telemetry is blunt about what happens when machine-speed output meets a human-speed review system: - ~**4×** the raw code output from daily AI users, for ~**12%** more delivered value (GitClear). - AI-coauthored PRs carry ~**1.7×** more issues; code churn up **861%**; per-developer defect rate up from **9% to 54%**; median review duration up **441%**; PRs merged with **zero review up 31%** (Faros AI, CodeRabbit). What to do about it: - **Tier review by blast radius, not by author.** A config change earns a glance; a payments path earns the full stack. - **Run *heterogeneous* AI reviewers.** In one study of 4 reviewers over 146 PRs, **93% of findings were caught by exactly one tool; none by all four.** Two *different* reviewers beat two copies of the best one. - **Raise the intake bar** — refuse to review changes that arrive without a stated purpose, a readable diff, and proof the tests ran. Keep PRs small by instruction. - **Read the test changes more carefully than the code** — agents "fix" failing tests by rewriting the assertion to match broken behavior. Mutation testing earns its place here. - **A human owns the merge.** A model can't be paged at 3am. Treat every AI review as a *sensor, not a verdict*. *Sources: Addy Osmani, ["Agentic Code Review"](https://addyosmani.com/blog/agentic-code-review/) (June 2026) and ["The Verification Bottleneck"](https://addyosmani.com/blog/verification-bottleneck/); Simon Willison, ["your job is to deliver code you have proven to work"](https://simonwillison.net/2025/Dec/18/code-proven-to-work/).* --- ## 5. Human *on* the loop (not *in* every turn) Because volume ended "a human reads every line," the human moves up a level rather than out. You stop reviewing every diff and start owning what doesn't transfer to a model: accountability, the judgment of whether this is even the right change, the high-blast-radius gates, and the behavior nobody specified. *Human-in-the-loop* becomes *human-on-the-loop*: sampling, spot-checking, and auditing the system, spending scarce attention where being wrong would actually hurt. *Source: ["Agentic Code Review"](https://addyosmani.com/blog/agentic-code-review/).* --- ## 6. Spec-Driven Development & Red/Green TDD A spec is no longer a prompt — it is product thinking made explicit, and the single highest-leverage instruction at the start of a task. Two disciplines matter more in a fleet: - **Write the spec for the *intent*, not the implementation** — goals, constraints, non-negotiables, and an explicit definition of *done* (fast, accessible, secure — beyond "functionally correct"). - **Red/green TDD** — write tests first, confirm they fail, then make them pass by correct implementation, not by gaming the test. With one developer, test-after is a manageable risk. With a fleet optimizing for "tests pass," test-after means a large surface of code whose tests confirm the wrong thing. *Source: ["The Factory Model"](https://addyosmani.com/blog/factory-model/).* --- ## 7. The three debts: Technical, Comprehension, Intent A clean way to think about software health (Storey's Triple Debt Model). The three are independent and bill you separately: - **Technical debt** lives in the **code**. Agents make it cheaper to take on *and* to pay down — point an agent at a tangled module and it refactors. - **Comprehension (cognitive) debt** lives in **people** — the growing gap between how much code exists and how much any human genuinely understands. Recoverable (ask the agent to explain), but invisible to velocity metrics until something breaks. One RCT: engineers who used AI to *generate* while learning scored **17% lower** on a comprehension quiz than those who used it for *inquiry*. - **Intent debt** lives in **artifacts** — the externalized *why*: goals, constraints, rationale. It's the one debt agents *can't* pay down, because intent is the one input that has to come from a human. An agent will invent a confident-sounding reason for that 300ms debounce, which is worse than admitting it doesn't know. Pay down intent debt by externalizing the why: specs for intent, `AGENTS.md` as an *intent ledger* (not autogenerated config), and lightweight decision logs (ADRs) captured at the moment you decide. *Sources: Addy Osmani, ["The Intent Debt"](https://addyosmani.com/blog/intent-debt/) (June 2026) and ["Comprehension Debt"](https://addyosmani.com/blog/comprehension-debt/) (Mar 2026).* --- ## 8. Cognitive Surrender (vs. Offloading) - **Cognitive offloading** — you delegate the *how* and still own the *what*. You judge whether the result is sensible and intervene when it isn't. A superpower. - **Cognitive surrender** — you stop constructing an answer at all; the model's output becomes yours and there's nothing left to check. In one study, when the AI was wrong, people accepted the wrong answer **73%** of the time — *and their confidence went up.* That's borrowed confidence. Moves that resist surrender: **construct an expectation before reading the output**; read the diff as if a junior wrote it ("seems right" is not a review); ask the model to argue against itself; notice when you're tired (surrender is a fatigue phenomenon); keep some solo time at the keyboard for calibration. *Source: Addy Osmani, ["Cognitive Surrender"](https://addyosmani.com/blog/cognitive-surrender/) (May 2026), on Shaw & Nave (Wharton).* --- ## 9. The Orchestration Tax Starting an agent is a keystroke; closing the loop on it is not. More agents running doesn't mean more of *you* — your judgment doesn't parallelize. You are the single-threaded resource (the "GIL" of your agents), and by Amdahl's Law throughput is capped by that serial fraction. Practical moves: **scale the fleet to your review rate, not the UI's spawn button** (backpressure); **batch reviews** to cut context-switch cost; **sort work** into delegate-able vs. judgment-heavy; **spend the lock only on judgment** and let the machine prove the boring 80% itself. *Source: Addy Osmani, ["The Orchestration Tax"](https://addyosmani.com/blog/orchestration-tax/) (May 2026).* --- ## 10. Self-Improving Loops (the "Ralph" technique) Autonomous loops that ship while you sleep: break work into atomic tasks with pass/fail criteria, run an agent in a loop (pick task → implement → validate → commit → log → reset context → repeat), and persist memory across iterations through four channels — **git history, a progress log, task state (e.g. `tasks.json`), and `AGENTS.md`**. Resetting context each iteration avoids drift; the memory files make each improvement make the next one easier (compounding). Guardrails are non-negotiable: feature branches not `main`, auto-approve read-only commands only, sandboxes, stop conditions, and a human PR review at the end. *Source: Addy Osmani, ["Self-Improving Coding Agents"](https://addyosmani.com/blog/self-improving-agents/) (Jan 2026).* --- ## How the concepts map to the dimensions | Dimension | Leans on | |-----------|----------| | Intent & Specification | Factory Model · Spec-Driven Dev · Intent Debt | | Context, Memory & Harness | Harness Engineering · Self-Improving Loops · Intent Debt | | Loops, Autonomy & Orchestration | Loop Engineering · Orchestration Tax · Self-Improving Loops | | Verification & Review | Agentic Code Review · Verification Bottleneck · Human-on-the-loop | | Tests & Deterministic Gates | Red/Green TDD · Verification Bottleneck · Factory Model | | Safety, Security & Guardrails | Harness Engineering (hooks/sandboxes) · Self-Improving Loops | | Comprehension & Judgment | Comprehension Debt · Cognitive Surrender | | Reusable Capability & the Ratchet | Harness Engineering (the ratchet) · Loop Engineering (skills) | | Measurement, Cost & Compounding | Orchestration Tax · Verification Bottleneck data · Self-Improving Loops | --- *All external claims and figures above are attributed to their sources. Quoted statistics reflect the cited 2026 reports and studies; verify against the primary source before using them in formal settings.*