Run 2026-09-13_x1_partner_control¶
Preregistered decision rule (PREREGISTRATION.md section 5)¶
- H1 (planted recall 9/12 vs decoy false positives 0/6, Fisher one-sided): p = 0.0045 -> PASS
- H2 (sampler enrichment, hypergeometric): -> FAIL
- H3 (S0 two-note recall vs B4 one-note recall, Fisher one-sided): p = 0.0196 -> PASS
Decision: NULL (signal requires H1 and H2 both passing).
Run summary¶
Total measured cost: $9.622686
Models:
- cards: claude-haiku-4-5-20251001 (alias haiku)
- critic: claude-haiku-4-5-20251001 (alias haiku)
- generator: claude-sonnet-5 (alias sonnet)
- match_gold: claude-haiku-4-5-20251001 (alias haiku)
Exploratory finds (non-planted survivors, manual review only): 3
Oracle recall on planted: 75.0% [47, 91]; permutation p = 0.0289
T1. Corpus
| notes | cards | cards_per_note | bridges | decoys | leakage_failures | corpus_sha256 |
|---|---|---|---|---|---|---|
| 60 | 324 | 5.4 | 12 | 6 | 0 | f17c4f086430 |
T2. Sampler enrichment; base rate 0.61% of 51603 cross-note card pairs are planted
| arm | draws | planted_hits | distinct_bridges | expected_hits | p_hypergeom |
|---|---|---|---|---|---|
T3. Oracle set: generator and critic without the sampler; mean cosine to gold on planted = 0.451
| S0 group | units | NONE rate | non-NONE rate | survivor rate (post critic + dupgate) |
|---|---|---|---|---|
| planted | 12 | 8.3% [1, 35] | 91.7% [65, 99] | 58.3% [32, 81] |
| decoy | 6 | 100.0% [61, 100] | 0.0% [0, 39] | 0.0% [0, 39] |
| recall on planted (gold match) | 12 | 75.0% [47, 91] |
T4. Per arm: NONE rate, critic kill rate, survivors, recovered planted bridges, measured cost
| arm | units | NONE | critic kill | dup | survivors | bridges reachable | bridges recovered | recall | cost usd | usd per recovered |
|---|---|---|---|---|---|---|---|---|---|---|
| B4 | 24 | 0.0% [0, 14] | 25.0% [12, 45] | 0 | 18 | 12 | 3 | 25.0% [9, 53] | 2.9705 | 0.9902 |
| S0 | 18 | 38.9% [20, 61] | 36.4% [15, 65] | 0 | 7 | 12 | 9 | 75.0% [47, 91] | 1.9919 | 0.2213 |
| S1 | 24 | 79.2% [60, 91] | 40.0% [12, 77] | 0 | 3 | 12 | 1 | 8.3% [1, 35] | 2.2891 | 2.2891 |
T5. Label-permutation null over the S0 oracle set
| statistic | observed | expected under null | p | shuffles |
|---|---|---|---|---|
| survivors among planted S0 units | 7 | 4.687 | 0.0289 | 10000 |
T6. Recombination vs single-note reflection on the same bridge notes
| arm | bridges reachable | recovered | recall | fisher p (S0 > B4) |
|---|---|---|---|---|
| S0 two notes | 12 | 9 | 75.0% [47, 91] | 0.0196 |
| B4 one note | 12 | 3 | 25.0% [9, 53] |
T7. Exploratory control (not preregistered): does the gold mechanism appear when the partner note is replaced by a mechanism-free filler from the same domain?
| arm | bridges reachable | recovered | recall | fisher p (S0 > S1) |
|---|---|---|---|---|
| S0 bridge note + true partner | 12 | 9 | 75.0% [47, 91] | 0.00138 |
| S1 bridge note + partner-domain filler | 12 | 1 | 8.3% [1, 35] | |
| B4 bridge note alone | 12 | 3 | 25.0% [9, 53] |