docs: declare run 2 answer rate as an outcome before it is known

The brief tells the model an empty predictions array is always available, so run 2 may answer fewer articles than run 1 did. Writing down now, while there are two run-2 proposals on the board, that selectivity gets reported as a result rather than quietly treated as a smaller sample. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WnNxwxfXSbeNtjvtz5gayb
2026-09-08 01:55:38 +01:00
parent b246bd9d4b
commit 20fb021983
1 changed files with 15 additions and 0 deletions
@@ -88,6 +88,21 @@ model the base rate will pull it toward the base rate. So:
 Beating run 1 while still sitting below 57.39% is not a system worth trading.
 That distinction gets reported every time, not just when it is convenient.
 ## Yield is an outcome too, declared before it is known
 The brief tells the model that an empty predictions array is always available
 and that it should prefer one on the families it reads worst. So run 2 may
 answer fewer than the 716 articles run 1 answered. Run 1's yield on this set is
 100% by construction, since these are precisely the articles it answered.
 Recorded now, with two run-2 proposals on the board and no idea what the rate
 will be: increased selectivity is a real behavioural change and gets reported as
 one, not quietly dropped for shrinking the sample. T4 runs on the shared
 articles whatever that number turns out to be. If the shared set falls below
 about 300 articles the paired test loses the power to see a 5 point shift, and
 the honest report is then "the feedback made it far more selective and the
 sample it left is too small to grade", not a null result dressed up as one.
 ## Housekeeping that is easy to forget
 Run 2 stays `running` once it exhausts its 716 articles, and the replay lane