Run 2026-09-14_x5_single_note_all¶
Preregistered decision rule (PREREGISTRATION.md section 5)¶
- H1 (planted recall 0/0 vs decoy false positives 0/0, Fisher one-sided): p = 1.0000 -> FAIL
- H2 (sampler enrichment, hypergeometric): -> FAIL
- H3: not computable (missing S0 or B4)
Decision: NULL (signal requires H1 and H2 both passing).
Run summary¶
Total measured cost: $15.046427
Models:
- cards: claude-haiku-4-5-20251001 (alias haiku)
- critic: claude-haiku-4-5-20251001 (alias haiku)
- critic_haiku: claude-haiku-4-5-20251001 (alias haiku)
- generator: claude-sonnet-5 (alias sonnet)
- match_gold: claude-haiku-4-5-20251001 (alias haiku)
Exploratory finds (non-planted survivors, manual review only): 0
Oracle recall on planted: 0.0% [0, 0]; permutation p = 1.0000
T1. Corpus
| notes | cards | cards_per_note | bridges | decoys | leakage_failures | corpus_sha256 |
|---|---|---|---|---|---|---|
| 96 | 484 | 5.04 | 16 | 12 | 13 | f8a6e9e33577 |
T2. Sampler enrichment; base rate 0.37% of 115829 cross-note card pairs are planted
| arm | draws | planted_hits | distinct_bridges | expected_hits | p_hypergeom |
|---|---|---|---|---|---|
T3. Oracle set: generator and critic without the sampler; mean cosine to gold on planted = 0.000
| S0 group | units | NONE rate | non-NONE rate | survivor rate (post critic + dupgate) |
|---|---|---|---|---|
| recall on planted (gold match) | 0 | 0.0% [0, 0] |
T4. Per arm: NONE rate, critic kill rate, survivors, recovered planted bridges, measured cost
| arm | units | NONE | critic kill | dup | survivors | bridges reachable | bridges recovered | recall | cost usd | usd per recovered |
|---|---|---|---|---|---|---|---|---|---|---|
| B4 | 92 | 1.1% [0, 6] | 24.2% [17, 34] | 0 | 69 | 16 | 6 | 37.5% [18, 61] | 11.5138 | 1.919 |
T5. Label-permutation null over the S0 oracle set
| statistic | observed | expected under null | p | shuffles |
|---|---|---|---|---|
| survivors among planted S0 units | 0 | 0.0 | 1.0000 | 2000 |
T8. Recall after critic, per critic (same generations, before the duplicate gate)
| critic | model id | correct recoveries | surviving | killed | decoy answers | decoy surviving | kill rate on random |
|---|---|---|---|---|---|---|---|
| haiku | claude-haiku-4-5-20251001 | 0 | 0 | 0 | 0 | 0 | 0.0% [0, 0] |