Run 2026-09-13_micro¶
Preregistered decision rule (PREREGISTRATION.md section 5)¶
- H1 (planted recall 8/12 vs decoy false positives 0/6, Fisher one-sided): p = 0.0113 -> PASS
- H2 (sampler enrichment, hypergeometric): B3 p = 0.482, B6 p = 1.000 -> FAIL
- H3 (S0 two-note recall vs B4 one-note recall, Fisher one-sided): p = 0.1102 -> FAIL
Decision: NULL (signal requires H1 and H2 both passing).
Run summary¶
Total measured cost: $33.620931
Models:
- cards: claude-haiku-4-5-20251001 (alias haiku)
- critic: claude-haiku-4-5-20251001 (alias haiku)
- generator: claude-sonnet-5 (alias sonnet)
- match_gold: claude-haiku-4-5-20251001 (alias haiku)
Exploratory finds (non-planted survivors, manual review only): 36
Oracle recall on planted: 66.7% [39, 86]; permutation p = 0.0320
T1. Corpus
| notes | cards | cards_per_note | bridges | decoys | leakage_failures | corpus_sha256 |
|---|---|---|---|---|---|---|
| 60 | 328 | 5.47 | 12 | 6 | 0 | f17c4f086430 |
T2. Sampler enrichment; base rate 0.65% of 52887 cross-note card pairs are planted
| arm | draws | planted_hits | distinct_bridges | expected_hits | p_hypergeom |
|---|---|---|---|---|---|
| B1 | 100 | 1 | 1 | 0.654 | 0.482 |
| B3 | 100 | 1 | 1 | 0.654 | 0.482 |
| B6 | 50 | 0 | 0 | 0.327 | 1 |
T3. Oracle set: generator and critic without the sampler; mean cosine to gold on planted = 0.428
| S0 group | units | NONE rate | non-NONE rate | survivor rate (post critic + dupgate) |
|---|---|---|---|---|
| planted | 12 | 8.3% [1, 35] | 91.7% [65, 99] | 41.7% [19, 68] |
| decoy | 6 | 83.3% [44, 97] | 16.7% [3, 56] | 0.0% [0, 39] |
| random | 42 | 81.0% [67, 90] | 19.0% [10, 33] | 14.3% [7, 28] |
| recall on planted (gold match) | 12 | 66.7% [39, 86] |
T4. Per arm: NONE rate, critic kill rate, survivors, recovered planted bridges, measured cost
| arm | units | NONE | critic kill | dup | survivors | bridges reachable | bridges recovered | recall | cost usd | usd per recovered |
|---|---|---|---|---|---|---|---|---|---|---|
| B1 | 100 | 84.0% [76, 90] | 12.5% [3, 36] | 0 | 14 | 1 | 0 | 0.0% [0, 79] | 8.7719 | n/a |
| B3 | 100 | 81.0% [72, 87] | 26.3% [12, 49] | 0 | 14 | 1 | 0 | 0.0% [0, 79] | 8.8493 | n/a |
| B4 | 24 | 0.0% [0, 14] | 25.0% [12, 45] | 0 | 18 | 12 | 4 | 33.3% [14, 61] | 2.8572 | 0.7143 |
| B6 | 50 | 96.0% [87, 99] | 0.0% [0, 66] | 0 | 2 | 0 | 0 | 0.0% [0, 0] | 4.0821 | n/a |
| S0 | 60 | 66.7% [54, 77] | 45.0% [26, 66] | 0 | 11 | 12 | 8 | 66.7% [39, 86] | 5.9546 | 0.7443 |
T5. Label-permutation null over the S0 oracle set
| statistic | observed | expected under null | p | shuffles |
|---|---|---|---|---|
| survivors among planted S0 units | 5 | 2.197 | 0.0320 | 10000 |
T6. Recombination vs single-note reflection on the same bridge notes
| arm | bridges reachable | recovered | recall | fisher p (S0 > B4) |
|---|---|---|---|---|
| S0 two notes | 12 | 8 | 66.7% [39, 86] | 0.11 |
| B4 one note | 12 | 4 | 33.3% [14, 61] |