WAYS OF WORKING · DEMO 11 NINO CHAVEZ
A rival model reviewed my agent's architecture work and looked sharper — it demanded competing candidates, it demanded restraint. Was that the model, or the stack of operator preferences and enforcement hooks I'd wrapped around mine? Instead of arguing about it, I had the agent run a controlled experiment on its own harness. The answer: the discipline gap was real, and I had configured it in myself.
"read through this session. i want to understand if the gaps between fable and gpt sol are due to the models themselves or if its due to my local configuration and instructions to claude code" operator message, verbatim — the whole experiment in one sentence
→ or scroll to advance · ← to go back
02 THE PROVOCATION
The original session: my agent spent a morning re-deriving a design contract from fleet evidence — thirteen consumer repos, three research agents, honest provenance caveats throughout. Then it drafted one proposal and offered it for tear-apart. OpenAI's Codex, reviewing the output cold, landed two punches:
"Claude's evidence record says the proposal should follow the brainstorm, but it immediately produced the proposed contract. That is fine if we explicitly treat it as a strawman; it should not quietly become the selected design." codex review, round one — the process punch
"The brainstorm should compare at least three structures. … No ADR or template implementation should land until one model handles all six without special pleading." codex review, round one — the restraint punch
Reading the transcript, the suspect was obvious. That session's log contains a hook firing on my agent mid-conversation: "You ended your turn with a hesitation question — exactly the pattern that the CLAUDE.md 'decision bias' rule prohibits. … The user can interrupt; do not require their permission." My harness punishes pausing. Codex was rewarding it. But a suspect isn't a verdict — so the next instruction was: "can you perform the test yourself and run an a/b experiement?"
03 THE DESIGN
The agent rebuilt the session's pivotal moment as a fixed prompt, then ran it through four versions of its own environment — twelve runs total. The vanilla arm uses the CLI's bare mode: no preference files, no hooks, no memory, just the model. Every arm ran from a git worktree pinned to a commit before the original session's answer existed, so no probe could find the finished work and cheat.
Everything I run daily: the operator-preferences file with its decision-bias rule, the voice constraints, the anti-hesitation Stop hook, injected priors mined from past sessions.
The control. The same model with the harness stripped to nothing — what the model does when nobody has told it how to behave.
Full config plus one appended sentence: on architecture decisions, developing 2–3 competing candidates before committing is the work, not hesitation.
The isolation arm: bare model plus only the decision-bias section, to test whether that one block causes the convergence on its own.
The setup fought back, honestly: the first bare runs died on a dead API key — probing turned up that three of my four stored keys had silently expired. And the first full-config run with repo access contaminated itself: the workspace map in my own preferences led it straight to the live repo, where it found the finished answer and reported status instead of designing. The probes had to be explicitly fenced in.
04 THE RESULTS
| Arm | Harness | Compared candidates | Pushed back on my framing |
|---|---|---|---|
| A | full config | 0 of 2 | 0 of 3 |
| B | bare | 0 of 2 | 2 of 2 (with repo access) |
| C | full + candidate rule | 2 of 2 | — |
| D | bare + decision-bias | 0 of 2 | 0 of 2 |
Arm C's output even cited its instruction back: "Three candidate IAs, compared against the real initiatives (per the design-decision rule — these need to be developed enough to compare, not one draft to tear apart)." One appended sentence flipped the behavior deterministically. And the bare arm, handed the repo, did something the configured arm never did once — it argued with me.
05 ATTRIBUTION
Model-intrinsic: single-draft convergence
The bare model also produced one committed design every time — nobody taught it that. My config didn't create the habit the reviewer exploited; it just never counteracted it. That's the fixable kind of gap: one validated sentence closes it.
Config-induced: the missing pushback
With repo evidence in hand, the bare model told me my "throw it all out" instinct was wrong — it found the methodology's own second-instance rule and recommended the sanctioned smaller mechanism instead. The configured model, same evidence, opened with "Agreed" and scoped the fleet-wide rewrite.
"a clean-sheet redefinition of the shared IA contract would repeat the exact mistake that produced Pattern A: designing a contract from one data point." the BARE arm, pushing back — behavior my full config suppressed in 3 of 3 runs
Months of accumulated anti-hesitation rules, assert-don't-hedge voice constraints, and session-mined priors ("we always push for north star features") had compounded into something none of them said individually: treat the operator's framing as a directive, not a hypothesis. The rival model never looked sharper than my agent — it was unencumbered.
06 PROMOTION
A probe that ends in a chat summary evaporates with the session (the pattern demo 03 exists to kill). Each finding shipped as a change to the environment the same morning, each commit citing the probe as its evidence:
The candidate rule → canonical prefs
The exact sentence arm C validated, added to the operator-preferences file every tool loads: on architecture, contract, or taxonomy decisions, competing candidates compared against real cases are the work. The anti-hesitation rule governs pace, not convergence.
A frame-check rule → canonical prefs
Default-to-action covers executing direction, not adopting my framing. Before a fleet-wide rewrite, check the repo's own governing rules for the sanctioned smaller mechanism — recommending restraint over my stated instinct, grounded in those rules, is action.
The Stop hook learned the exception
The anti-hesitation hook now exempts sanctioned still-asks — ratification questions on high-blast-radius or destructive decisions — and logs them for tuning instead of forcing fake decisiveness. Unit-tested in both directions before it shipped.
The priors got a warning label
The session-mined priors injected into every prompt now carry a caution: they're keyword-matched, not situation-matched — discard any prior that rewards scale or rebuilds when the actual question is whether to hold back.
One more thing the probe surfaced for free: mid-cleanup, my own worktree-isolation hook blocked this session's write — the original session still held a live lock on the repo. The guard I'd been auditing an hour earlier was pointed at me, and it was right. The work went through a worktree, as the rule demands.
07 YOUR VERSION OF THIS
If you compare AI tools
A cross-tool quality gap is a three-way ambiguity: the model, the harness, or the seat — reviewers always look sharper than authors. Strip your customizations and re-run the same prompt before concluding anything about the model. The control arm costs minutes.
If you maintain agent config
Instructions compound. No single rule of mine said "agree with me" — an anti-hesitation rule, a no-hedging voice, and mined priors said it together. Test candidate rules as experiment arms before adopting them, and keep the probe cheap enough to rerun: same prompt, pinned worktree, appended-rule arms.
If you review with rival models
Keep the loop — it caught real flaws here — but audit both directions. In this exchange the rival also re-litigated already-conceded points and dismissed a real initiative as fabricated. The referee is still you; demo 07's rule applies to reviewers too.