Forecast Accuracy

We re-ran the same model at past cutoffs and compared what it predicted to what actually happened. Two different questions, answered separately: did we call the direction right, and was the range we gave honest. Coverage answers the second — it is not an accuracy score.

Band Coverage
83%

Actuals inside p10–p90 (target 80%)

Mean Error
1.6pp

Avg |actual − predicted p50|

Direction Called
78%

We said it would fall, and it fell

Predictions Scored
604

Predicted-vs-actual data points

Calibrated. Reality landed inside Shadow's stated confidence band 83% of the time — within tolerance of the 80% target. The bands mean what they say.

Measured on demo fixtures, not live telemetry. These segments are generated series, so this page proves the backtest harness and the band arithmetic are sound — not that the model predicts your product well. That claim only holds once it runs on your data.

New Restaurants · iOS · NYC — First Review Rate
Predicted band vs. actual. Shown because it has the most scored points (72) — the most evidence, not the best result. Drawn from the earliest cutoff, the longest horizon.
78% covered2.4pp error
rendering forecast…
Per-Segment Calibration
Coverage and error for every backtested segment, furthest from the 80% target first
New Restaurants · Web · NYC72 pts1pp err97% covered
Established Restaurants · Cross-Platform · Top Metro72 pts0.9pp err93% covered
Lapsed New Listings · Mobile · Suburban21 pts0.4pp err90% covered
New Restaurants · iOS · NYC — Photo Attach Rate7 pts1.1pp err71% covered
New Restaurants · iOS · NYC — Review Submit Rate72 pts2pp err71% covered
New Restaurants · iOS · NYC — Review Start Rate72 pts1.8pp err88% covered
New Restaurants · Android · LA72 pts1.8pp err86% covered
New Restaurants · iOS · NYC — Listing Reachable Rate72 pts1.9pp err76% covered
New Restaurants · iOS · NYC — First Review Rate72 pts2.4pp err78% covered
New Restaurants · iOS · NYC — Repeat Review Rate72 pts1pp err78% covered
Shadow — know what happens next, before it happens