fix: the outcome worker was starving its own queue
Separate bug from the cache, same symptom. The candidate query approximated maturity with '+horizon_days days', which is looser than the truth because 20 trading days is 28 calendar days. So the oldest hundred open rows were almost all horizon-30s that were not due yet, 98 of them, and those slots do not free up for weeks. Measured just now: 54 live predictions genuinely ready to score, zero of them inside the window. The worker sat there logging nothing at all. It only looked like it was working before because the three rows it could see were being retired as unresolvable every poll, which freed the slots. Fixing the cache stopped the retiring and revealed the stall underneath. 1.5x + 2 calendar days overshoots a trading day span at every horizon in use, so the limit now gets spent on rows that can plausibly be scored. The js trading day filter is still what decides. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -88,15 +88,22 @@ async function resolveAutonomyOutcomes({ intelligencePath, workerId = `outcome-$
|
||||
const cache = new Map();
|
||||
const failures = new Map();
|
||||
while (true) {
|
||||
// The sql filter is deliberately loose, it only counts calendar days and cannot
|
||||
// know about weekends or when a close actually publishes. Trading day
|
||||
// arithmetic, the same arithmetic calculateOutcome uses to find the exit bar,
|
||||
// then decides what is genuinely ready.
|
||||
// The sql filter cannot do trading day arithmetic, so it approximates and the
|
||||
// js filter below is still the authority on what is genuinely ready.
|
||||
//
|
||||
// It used to approximate the other way, '+horizon_days days', which is LOOSER
|
||||
// than the truth since 20 trading days is 28 calendar days. The oldest 100 open
|
||||
// rows were then almost all not-yet-due horizon-30s, 98 of them, and they sit
|
||||
// there for weeks. 54 live predictions that were genuinely ready never got into
|
||||
// the window at all and the worker quietly settled at zero scored per poll.
|
||||
// 1.5x + 2 overshoots a trading day span at every horizon we use, so the limit
|
||||
// gets spent on rows that can plausibly be scored. Costs a day or two of lag on
|
||||
// the long horizons, which is a fair trade for not starving.
|
||||
const candidates = db.prepare(`
|
||||
SELECT p.* FROM autonomy_predictions p
|
||||
LEFT JOIN autonomy_outcomes o ON o.prediction_id = p.id
|
||||
WHERE p.status = 'open' AND o.prediction_id IS NULL
|
||||
AND datetime(p.information_cutoff, '+' || p.horizon_days || ' days') <= datetime('now')
|
||||
AND datetime(p.information_cutoff, '+' || CAST(p.horizon_days * 1.5 + 2 AS INTEGER) || ' days') <= datetime('now')
|
||||
ORDER BY p.information_cutoff ASC LIMIT 100
|
||||
`).all();
|
||||
|
||||
|
||||
Reference in New Issue
Block a user