Run 2026-09-13_x3_cross_domain_near¶
Preregistered decision rule (PREREGISTRATION.md section 5)¶
- H1 (planted recall 0/0 vs decoy false positives 0/0, Fisher one-sided): p = 1.0000 -> FAIL
- H2 (sampler enrichment, hypergeometric): B3 p = 1.000 -> FAIL
- H3: not computable (missing S0 or B4)
Decision: NULL (signal requires H1 and H2 both passing).
Run summary¶
Total measured cost: $29.512699
Models:
- cards: claude-haiku-4-5-20251001 (alias haiku)
- critic: claude-haiku-4-5-20251001 (alias haiku)
- critic_haiku: claude-haiku-4-5-20251001 (alias haiku)
- generator: claude-sonnet-5 (alias sonnet)
- match_gold: claude-haiku-4-5-20251001 (alias haiku)
Exploratory finds (non-planted survivors, manual review only): 39
Oracle recall on planted: 0.0% [0, 0]; permutation p = 1.0000
T1. Corpus
| notes | cards | cards_per_note | bridges | decoys | leakage_failures | corpus_sha256 |
|---|---|---|---|---|---|---|
| 60 | 323 | 5.38 | 12 | 6 | 0 | f17c4f086430 |
T2. Sampler enrichment; base rate 0.61% of 51284 cross-note card pairs are planted
| arm | draws | planted_hits | distinct_bridges | expected_hits | p_hypergeom |
|---|---|---|---|---|---|
| B1 | 100 | 1 | 1 | 0.614 | 0.46 |
| B3 | 100 | 0 | 0 | 0.614 | 1 |
| B7 | 100 | 2 | 2 | 0.614 | 0.126 |
T3. Oracle set: generator and critic without the sampler; mean cosine to gold on planted = 0.000
| S0 group | units | NONE rate | non-NONE rate | survivor rate (post critic + dupgate) |
|---|---|---|---|---|
| recall on planted (gold match) | 0 | 0.0% [0, 0] |
T4. Per arm: NONE rate, critic kill rate, survivors, recovered planted bridges, measured cost
| arm | units | NONE | critic kill | dup | survivors | bridges reachable | bridges recovered | recall | cost usd | usd per recovered |
|---|---|---|---|---|---|---|---|---|---|---|
| B1 | 100 | 84.0% [76, 90] | 18.8% [7, 43] | 0 | 13 | 1 | 0 | 0.0% [0, 79] | 9.1077 | n/a |
| B3 | 100 | 91.0% [84, 95] | 22.2% [6, 55] | 0 | 7 | 0 | 0 | 0.0% [0, 0] | 8.7001 | n/a |
| B7 | 100 | 69.0% [59, 77] | 32.3% [19, 50] | 0 | 21 | 2 | 1 | 50.0% [9, 91] | 9.4717 | 9.4717 |
T5. Label-permutation null over the S0 oracle set
| statistic | observed | expected under null | p | shuffles |
|---|---|---|---|---|
| survivors among planted S0 units | 0 | 0.0 | 1.0000 | 10000 |
T8. Recall after critic, per critic (same generations, before the duplicate gate)
| critic | model id | correct recoveries | surviving | killed | decoy answers | decoy surviving | kill rate on random |
|---|---|---|---|---|---|---|---|
| haiku | claude-haiku-4-5-20251001 | 0 | 0 | 0 | 0 | 0 | 0.0% [0, 0] |