Skip to content

Run 2026-09-14_micro_v0_2

Preregistered decision rule, v0.2 (PREREGISTRATION-v0.2.md section 5)

  • H1 (planted recall 6/16 vs decoy false positives 0/12, Fisher one-sided): p = 0.0213 -> PASS
  • H2a (B7 planted card pairs 2 in 300 draws, expected 1.09, hypergeometric): p = 0.2973; arm-label permutation vs B1: gap = +0.0000, p = 0.6929 -> FAIL
  • H2b (non-NONE rate B7 58/300 vs B1 47/300, Fisher one-sided): p = 0.1413 -> FAIL
  • H3 (S0 two-note recall vs B4 one-note recall, Fisher one-sided): p = 0.8574 -> FAIL
  • H4 (S0 recall 6/16 vs S1 partner-domain-filler recall 0/16, Fisher one-sided): p = 0.0088 -> PASS
  • H5 (recall by writer family, estimation only, no pass/fail):
  • A: 3/10 = 30.0% [11, 60]
  • B: 3/6 = 50.0% [19, 81]

Decision: SIGNAL (signal requires H1 and H4 both passing; H2, H3, H5 are reported and do not enter the rule).

Run summary

Total measured cost: $85.038388

Models: - cards: claude-haiku-4-5-20251001 (alias haiku) - critic: claude-haiku-4-5-20251001 (alias haiku) - critic_haiku: claude-haiku-4-5-20251001 (alias haiku) - critic_sonnet: claude-sonnet-5 (alias sonnet) - generator: claude-sonnet-5 (alias sonnet) - match_gold: claude-haiku-4-5-20251001 (alias haiku)

Exploratory finds (non-planted survivors, manual review only): 75

Oracle recall on planted: 37.5% [18, 61]; permutation p = 0.0002

T1. Corpus

notes cards cards_per_note bridges decoys leakage_failures corpus_sha256
96 483 5.03 16 12 13 f8a6e9e33577

T2. Sampler enrichment; base rate 0.36% of 115345 cross-note card pairs are planted

arm draws planted_hits distinct_bridges expected_hits p_hypergeom
B1 300 2 2 1.09 0.297
B7 300 2 2 1.09 0.297

T3. Oracle set: generator and critic without the sampler; mean cosine to gold on planted = 0.547

S0 group units NONE rate non-NONE rate survivor rate (post critic + dupgate)
planted 16 43.8% [23, 67] 56.2% [33, 77] 31.2% [14, 56]
decoy 12 83.3% [55, 95] 16.7% [5, 45] 0.0% [0, 24]
random 36 91.7% [78, 97] 5.6% [2, 18] 0.0% [0, 10]
recall on planted (gold match) 16 37.5% [18, 61]

T4. Per arm: NONE rate, critic kill rate, survivors, recovered planted bridges, measured cost

arm units NONE critic kill dup survivors bridges reachable bridges recovered recall cost usd usd per recovered
B1 300 84.3% [80, 88] 21.3% [12, 35] 0 37 2 0 0.0% [0, 66] 26.9307 n/a
B4 32 3.1% [1, 16] 22.6% [11, 40] 0 24 16 8 50.0% [28, 72] 3.917 0.4896
B7 300 80.7% [76, 85] 36.2% [25, 49] 0 37 2 0 0.0% [0, 66] 27.5973 n/a
S0 64 78.1% [67, 86] 61.5% [36, 82] 0 5 16 6 37.5% [18, 61] 6.2552 1.0425
S1 32 90.6% [76, 97] 50.0% [9, 91] 0 1 16 0 0.0% [0, 19] 2.7252 n/a

T5. Label-permutation null over the S0 oracle set

statistic observed expected under null p shuffles
survivors among planted S0 units 5 1.256 0.0002 10000

T6. Recombination vs single-note reflection on the same bridge notes

arm bridges reachable recovered recall fisher p (S0 > B4)
S0 two notes 16 6 37.5% [18, 61] 0.857
B4 one note 16 8 50.0% [28, 72]

T7. Exploratory control (not preregistered): does the gold mechanism appear when the partner note is replaced by a mechanism-free filler from the same domain?

arm bridges reachable recovered recall fisher p (S0 > S1)
S0 bridge note + true partner 16 6 37.5% [18, 61] 0.00884
S1 bridge note + partner-domain filler 16 0 0.0% [0, 19]
B4 bridge note alone 16 8 50.0% [28, 72]

T8. Recall after critic, per critic (same generations, before the duplicate gate)

critic model id correct recoveries surviving killed decoy answers decoy surviving kill rate on random
haiku claude-haiku-4-5-20251001 6 3 3 2 0 100.0% [34, 100]
sonnet claude-sonnet-5 6 5 1 2 1 100.0% [34, 100]