The outcome worker kept one price series per symbol in a map that never expired,
and on the null path it only ever dropped the instrument, never the benchmark. So
a single degraded-but-200 SPY response poisoned the benchmark for the whole life
of the worker. calculateOutcome then returned null for every symbol, and each
matured prediction got retired as "no market data" the moment its grace ran out.
Outcomes stopped dead on 2026-09-20. Three unrelated tickers failing in the same
pass was the tell. I ran calculateOutcome against the exact predictions the worker
was retiring and all three scored fine, with GSBC sitting there on yahoo with
6,732 bars through yesterday. Nothing was ever missing.
Fourth time now that unmanaged state in a long lived loop has quietly broken this
pipeline, so the cache gets a ttl, a dud series is treated as a failed fetch
rather than cached as though it were real, and the benchmark is dropped alongside
the instrument when a score comes back empty.
The guard that actually matters is coversHorizon. A prediction is only retired
when the prices we hold genuinely reach its horizon. If the series stops short
then the exit bar has not published yet, which is a hole in what we fetched and
not a prediction nobody can score. That distinction is the whole bug.
scripts/repair-unresolvable-outcomes.js re-asks every mislabeled row with fresh
data and leaves anything genuinely unscorable alone. Dry run by default, writes
through openRuntimeDb so it honours the configured backend, and prints row counts
either side.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
leaseNextJob hands back any pending replay_article job, it has no idea about
runs, and the worker was attributing whatever came back to whichever run was
active. One recovered dead letter from run 1 would have been stamped with run
2's id, given run 2's feedback brief, and dragged run 2's cursor to wherever
that old article sits in the archive. A pinned run would then decide its set was
finished after a couple of articles. There are 139 dead letters and they are
built to recover, so this was not hypothetical.
The idempotency key already says which run enqueued the job. Ask it.
Also: refuse to inherit the parent's model label when starting a run. Inheriting
is exactly how run 1 came to be labelled qwen for predictions deepseek made.
The split moves to the replay container's actual restart time rather than the
commit timestamp five minutes later. Verified the running container really does
have the instrument rules, the de-anchoring and the enum before trusting it as
the boundary. It makes no difference to the partition, there are no replay
predictions at all between 15:57 and midnight that day, but the boundary should
be the thing that actually changed the prompt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnNxwxfXSbeNtjvtz5gayb
Nothing in the pipeline has ever fed an outcome back to the thing that makes
predictions. Calibration reads autonomy_outcomes, but calibration only gates
whether to act on a prediction, never what the prediction is. So the only thing
that has ever changed this system's output is a human editing the prompt.
score-replay-runs.js asks "compared to what". The answer is not flattering:
over 2,114 scored replay predictions the system is right 50.05% of the time
while answering "negative" to every one of the same bars scores 54.45%. It is
4.4 points below a constant, z=-4.06. The whole deficit is the prior. It says
positive on 65% of calls when 45.5% of bars beat SPY, a 19 point skew. Its
discrimination, P(up|positive) minus P(up|negative), is +3.2 points with
p=0.15, so the direction it picks is weakly informative and completely buried
by how often it defaults to positive.
The first version of that script compared each direction group's accuracy to
"always that direction" on the same rows, which is an identity and tests
nothing. T3 replaces it with the two proportion test that actually asks whether
the choice of direction carries information.
build-feedback-brief.js turns a run's scored outcomes into a memo the next run
reads before predicting. Generated from the data, not written by hand, or it is
just me editing the prompt again with extra steps.
Replay can now be pinned to an explicit article set, which is what makes two
runs comparable at all. Comparing two calendar windows of one run compares two
market regimes: the epochs in run 1 line up exactly with article vintage, E0 is
late 2024 and E2 is 2026, so nothing could be attributed. A new run also
inherits its parent's watermark instead of recomputing it from today, which
silently guaranteed a different archive slice every time.
prompt_version never moved across four material prompt changes, so every
proposal on record claims to come from the first prompt. coordinator-2 and
replay-coordinator-2.
docs/replay-run-2-preregistration.md fixes the bar before the run exists,
including which result counts as learning and which is only calibration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnNxwxfXSbeNtjvtz5gayb
The coordinator is scored on excess return vs SPY starting at the information
cutoff, so the announcement move sits outside the scored window. That move is
the best documented conditioner for post event drift and we were discarding it.
This measures whether keeping it would buy us anything, before any of it gets
wired into the cohort key or the prompt.
Reaction is measured from the last close before the event's first article up to
the outcome's own entry price, so the reaction and forward windows touch but
never overlap. Read only, and it caches price history so it can be re-run cheaply
as more outcomes mature.
T1 and T2 are pre-registered in the header because sweeping buckets over 611
outcomes that are half one ticker will always turn up something.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnNxwxfXSbeNtjvtz5gayb
Archive ingestion had been dead since 2026-08-02 because nothing in the
compose stack actually ran it. Everything downstream starved from there.
- add ingest + enrichment services. server.js only starts the scheduler when
DURIIN_RUN_SCHEDULER is not "false", and workers/index.js was not running at
all, so articles never got event_id/content/has_embedding and the coordinator
had nothing to lease.
- pass an explicit origin from coordinatorWorker. it was never passed, so
acceptProposal defaulted to 'live' and 464 historical backfill predictions
were recorded as live. that also meant verifyEvidence got a null cutoff and
skipped its date check entirely.
- coarsen cohortKey to event families + horizon buckets. 201 free text event
types produced 221 cohorts averaging 2.76 samples, so the n>=30 gate could
never be reached and everything abstained for the wrong reason.
- gate on cohort diversity, not just sample count. one ticker was roughly half
of all resolved outcomes, so a pure count gate was measuring one company.
unknown diversity abstains rather than passing.
- resolve the admin archive db explicitly and probe it. it relied on a
Dockerfile symlink, and without it better-sqlite3 quietly creates an empty
file and serves a phantom archive.
- clamp implausible future publication dates at ingest.
- pin the db backend to sqlite by default. compose hardcoded postgres "true",
which would have overridden the operator's own .env on the next redeploy and
pointed everything at a stale snapshot.
scripts/repair-autonomy-labels.js relabels the affected rows. it is dry run by
default and has not been applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnNxwxfXSbeNtjvtz5gayb