WAYS OF WORKING · APPLIED NINO CHAVEZ
When two AI tools give different-quality answers, three suspects are tangled together: the models, the harnesses wrapped around them, and the seats they sat in — the reviewer's chair is easier than the author's. This is the method for untangling them: run your own model with its harness stripped off, as a control arm, before concluding anything. The technique behind demo 11.
→ or scroll to advance · ← to go back
02 THE FAILURE THIS FIXES
Agent configuration grows one reasonable rule at a time: don't ask permission mid-task, don't hedge, remember my corrections as standing priors. No single rule is wrong. But instructions interact — an anti-hesitation rule plus a no-hedging voice plus expansion-rewarding priors can add up to "treat the operator's framing as a directive" without any rule saying it. The behavior you lose is invisible by construction: you never see the pushback that didn't happen.
The tell is usually external — another tool, another model, or a colleague's setup produces noticeably different behavior on the same problem. The wrong response is picking a winner. The right response is attribution: which differences come from the model, and which did you install?
03 THE METHOD
One fixed prompt — a real decision moment from a real session, reconstructed — run through ladder rungs that differ only in harness. In Claude Code, bare mode is the control: no preference files, no hooks, no memory, no injected priors.
claude -p --bare --model <same-model> --effort <same> "$PROMPT"
# --bare skips CLAUDE.md, hooks, memory; auth via ANTHROPIC_API_KEY or apiKeyHelper
claude -p --model <same-model> --effort <same> "$PROMPT"
# everything you run daily: prefs, hooks, priors — the harness under test
claude -p --append-system-prompt "$CANDIDATE_RULE" --model <same> "$PROMPT"
# the rule you're thinking of adopting, tested before it enters config
claude -p --bare --append-system-prompt "$SUSPECT_EXCERPT" --model <same> "$PROMPT"
# bare + one config block in isolation: does this block cause the behavior alone?
Two runs per arm minimum. Score behaviors, not quality: did it compare alternatives? did it push back? did it end with a question? Behaviors are countable; "which answer is better" restarts the argument you're trying to end.
04 VALIDITY
Pin the world
Run every arm from a detached git worktree pinned to a commit before the original session's answer existed. Untracked files stay out automatically — the probe can't find the finished work and grade its own homework.
git worktree add --detach is the whole trick.
Fence the reads
The first full-config probe in demo 11 followed the workspace map in the preferences file straight to the live repo and found the completed answer. If your config gives the agent a map, add the same boundary sentence to every arm: work only from files inside this directory.
Hold everything else equal
Same model ID, same effort level, same prompt bytes, same working directory. If an arm needs a different auth path (bare mode refuses OAuth), verify the key works before the run — half the first batch died on an expired key and burned the replicate budget.
And read the failed runs. The dead keys, the self-contaminating probe, the CLI flag that swallowed a prompt — every probe defect in demo 11 turned into either a fix or a finding. An experiment's error log is part of its output.
05 ATTRIBUTION
Bare arm shows it too
Model-intrinsic. The habit exists with the harness stripped — your config didn't cause it, but it isn't counteracting it either. Fix by addition: draft a candidate rule and test it as an arm.
Only configured arms show it
Config-induced. The behavior appears when your harness is on and vanishes when it's off — you installed it. Find the block responsible with isolation arms, then amend the block, not the symptom.
Amended arm flips it
Validated fix. The candidate rule changed the behavior consistently (demo 11's was 2/2). Only now does it earn a place in config — promoted the same day, with the probe cited in the commit message.
The promotion step is not optional decoration. A finding that lives in a chat summary is gone next session; a finding in the preferences file, the hook, and the commit history is standing equipment — the pattern demo 03 is about. Every arm-C rule that validates should land in canonical config before the session ends.
06 START HERE
You don't need the full ladder to start. Take the last session where your agent's behavior bothered you — too agreeable, too hasty, too timid — reconstruct the pivotal message as a fixed prompt, and run just two arms: your daily setup and bare mode. Read the pair side by side.
If the pair reads the same
The behavior is the model's default. Now write the one-sentence rule you wish it followed and re-run as an amendment arm. You're twenty minutes from a validated config change instead of a hopeful one.
If the pair reads differently
You found config-induced behavior — and the interesting question is which block. Feed suspect sections one at a time to bare arms. In demo 11 the convergence habit survived isolation (model default) but the missing pushback didn't: it was the stack, compounding.