5 The System in Action
With setup covered, here is what the system actually does. This section walks through the three core mechanisms that make the workflow powerful: specialized agents, adversarial QA, and automatic quality scoring.
5.1 Why Specialized Agents Beat One-Size-Fits-All
Consider proofreading a 140-slide lecture deck. You could ask Gemini:
“Review these slides for grammar, layout, math correctness, code quality, and pedagogical flow.”
Gemini will skim everything and catch some issues. But it will miss:
- The equation on slide 42 where a subscript changed from \(m_t^{d=0}\) to \(m_t^0\)
- The TikZ diagram where two labels overlap at presentation resolution
- The R script that uses
k=10covariates but the slide saysk=5
Now compare with specialized agents:
| Agent | Focus | What It Catches |
|---|---|---|
proofreader |
Grammar only | “principle” vs “principal” |
slide-auditor |
Layout only | Text overflow on slide 37 |
pedagogy-reviewer |
Flow only | Missing framing sentence before Theorem 3.1 |
r-reviewer |
Code only | Missing set.seed() |
domain-reviewer |
Substance | Slide says 10,000 MC reps, code runs 1,000 |
Each agent reads the same file but examines a different dimension with full attention. The slide-excellence skill runs them all in parallel.
5.2 The Adversarial Pattern: Critic + Fixer
The single most powerful pattern in this system is the adversarial QA loop:
+------------------+
| quarto-critic | "I found 12 issues. 3 Critical."
| (READ-ONLY) |
+--------+---------+
|
+----v----+
| Verdict |
+----+----+
/ \
APPROVED NEEDS WORK
| |
Done +----v---------+
| quarto-fixer | "Fixed 12/12 issues."
| (READ-WRITE) |
+----+---------+
|
+----v----------+
| quarto-critic | "Re-audit: 2 remaining."
| (Round 2) |
+----+----------+
|
... (up to 5 rounds)
Why it works: The critic can’t fix files (read-only), so it has no incentive to downplay issues. The fixer can’t approve itself (the critic re-audits). This prevents the common failure of Gemini saying “looks good” about its own work.
The critic-fixer loop above is text-only — it compares LaTeX source to Quarto source. It can’t see the rendered slides. Google’s Multimodal Vision capabilities let Gemini accept rendered slide images (such as PDF slides converted to PNGs) directly into the agent context to verify layouts visually. If you want closed-loop visual QA — “does the rendered slide actually look right?” — this multimodal capability closes the gap.
Optional, not required: qa-quarto and visual-audit already catch most issues without it. Treat visual verification as an extra rung when text-level audits aren’t enough (e.g., suspecting a TikZ render glitch that the source code looks fine for).
5.3 The Orchestrator: A Pattern, Not a Daemon
Individual agents are specialists. Skills like slide-excellence and qa-quarto coordinate a few agents for specific tasks. In day-to-day work, you don’t have to think about which agents to run — the right skill runs the right agents for you.
The orchestrator protocol (.agents/rules/orchestrator-protocol.md) defines the 6-step loop (IMPLEMENT → VERIFY → REVIEW → FIX → RE-VERIFY → SCORE) that skills implement internally. It is not a runtime daemon: plan approval does NOT auto-trigger this loop. You invoke a skill (e.g., create-lecture, qa-quarto, review-paper --adversarial) and the skill runs the pattern within its own scope.
What’s mechanically implemented today: commit (verifier + quality_score), qa-quarto (critic-fixer loop), review-paper --adversarial (critic-fixer loop), slide-excellence (multi-agent fanout), review-paper --peer (editor + 2 referees + cross-artifact). See Pattern 2 for the complete workflow.
5.4 Quality Review: The 80/90/95 System
The quality-gates rule (quality-gates.md) defines scoring thresholds that commit and review skills apply. Thresholds are advisory — enforced inside specific skills, not by a repo-wide git pre-commit hook. A direct git commit bypasses the review. Every file gets a quality score from 0 to 100:
| Score | Threshold | Meaning | Action |
|---|---|---|---|
| 80+ | Commit | Safe to save progress | git commit allowed |
| 90+ | PR | Ready for deployment | gh pr create encouraged |
| 95+ | Excellence | Exceptional quality | Aspirational target |
| < 80 | Blocked | Critical issues exist | Must fix before committing |
5.4.1 How Scores Are Calculated
Points are deducted for issues:
| Issue | Deduction | Why Critical |
|---|---|---|
| Equation overflow | -20 | Math cut off = unusable |
| Broken citation | -15 | Academic integrity |
| Equation typo | -10 | Teaches wrong content |
| Text overflow | -5 | Content cut off |
| Label overlap | -5 | Diagram illegible |
| Notation inconsistency | -3 | Student confusion |
5.4.2 Mandatory Verification
The verification protocol (.agents/rules/verification-protocol.md) requires that Gemini compile, render, or otherwise verify every output before reporting a task as complete. Skills that implement the orchestrator pattern enforce this as Step 2: VERIFY. This means Gemini cannot say “done” from within those skills without actually checking the output.
Verification catches errors that might silently deploy, like broken diagrams or missing intercepts in scripts.
AI providers regularly update model checkpoints and release system changes. These updates can sometimes lead to model-quality drift or subtle regressions on highly specialized tasks. Implication for our workflow: do not treat any given model checkpoint as a stable baseline.
The defences in this template all assume model quality drifts:
verify-claimswith the Chain-of-Verification forked-verifier (the verifier cannot self-confirm even if the orchestrator’s checkpoint regressed).audit-reproducibilitywithpassport.yaml(numeric claims are anchored to specific script output values, not to the model’s recall of the values).- The cross-artifact review rule (a paper’s claims are checked against the code that produced them, not against the model’s intuition about whether they’re “plausible”).
- HIGH-WARN gate-refuse on
commit(Pass 3A I) — a fabricated citation or numerical contradiction blocks the commit even if the model “feels confident.”
This is also a useful reminder that running review-paper --variance N (Pass 2C E) is the empirical answer to “should I trust this output?” — a single point estimate of model quality on a single task hides variance that the template’s adversarial-review patterns surface.
5.4.3 Submission-readiness: stacking the verification lenses
For a paper headed to a journal, three orthogonal lenses run together — each catches a different class of failure:
| Lens | Skill | What it catches | Block-commit on failure? |
|---|---|---|---|
| Grammar / overflow | proofread |
typos, search-and-replace artifacts, overfull \hbox, citation-format inconsistency |
No (advisory) |
| AI-voice tells (v1.9.0) | humanize |
boilerplate transitions (“Moreover”, “It is important to note”), AI-cliché lexicon (“delve”, “navigate the complexities”), hedging stacking, sycophancy | No (advisory; author edits manually) |
| Factual claims | verify-claims |
fabricated citations, numerical contradictions, directional contradictions | Yes — HIGH-WARN gate-refuses (v1.9.0) |
| Numeric provenance | audit-reproducibility |
manuscript value ≠ script output value within tolerance | Yes via passport.yaml (v1.9.0) |
These are complementary, not redundant. A paper can pass proofread (clean grammar) and fail humanize (the prose reads as AI-drafted). It can pass humanize (your own voice) and fail verify-claims (citation fabricated). It can pass verify-claims (citations real) and fail audit-reproducibility (Table 2 doesn’t match the code). Run all four before submission; reach for review-paper --variance N for the simulated peer-review verdict on top.
humanize design choice: detect-only, no --rewrite mode. Auto-rewriting AI tells degrades prose quality (cross-vendor research finding) and introduces new tells. The author reads the report and edits — that manual step is the price of preserving voice. If the report flags 8+ HIGH-severity tells per 1000 words, rewrite the affected paragraph from scratch rather than patching tell-by-tell.
5.5 Creating Your Own Domain Reviewer
The template includes domain-reviewer.md — a skeleton for building a substance reviewer specific to your field.
5.5.1 The 5-Lens Framework
Every domain can benefit from these five review lenses:
| Lens | What It Checks | Example (Economics) | Example (Political Science) | Example (Physics) |
|---|---|---|---|---|
| Assumption Audit | Are stated assumptions sufficient? | Is overlap required for ATT? | Is ignorability defensible given the observed covariates? | Is the adiabatic approximation valid here? |
| Derivation Check | Does the math check out? | Do decomposition terms sum? | Do conjoint AMCEs identify under Hainmueller–Hopkins–Yamamoto assumptions? | Do the units balance? |
| Citation Fidelity | Do slides match cited papers? | Is the theorem from the right paper? | Is the manipulation-check threshold cited from the original validation study? | Is the experimental setup correctly described? |
| Code-Theory Alignment | Does code implement the formula? | R script matches the slide equation? | cjoint/survey::svyglm weights match the design? |
Simulation parameters match theory? |
| Logic Chain | Does the reasoning flow? | Can a PhD student follow backwards? | Does the causal claim survive the standard counterfactual challenge? | Are prerequisites established? |
The template ships two concrete domain-reviewer customizations in .agents/agents/domain-reviewer.md: an econometrics example (assumptions, identification, R/Stata code-theory alignment) and a political-science example (ignorability, conjoint AMCEs, cjoint/survey package defaults). Both follow the 5-lens structure; either is a viable starting point for your own field.
To customize, open .agents/agents/domain-reviewer.md and fill in:
- Your domain’s common assumption types
- Typical derivation patterns to verify
- Key papers and their correct attributions
- Code-theory alignment checks for your tools
- Logic chain requirements for your audience