What we did
- Collected timestamped news and market-style questions with later-resolved outcomes.
- Used the known future outcome at each question's resolution date as the training label — no annotators.
- Trained Foresight-32B with RL (GRPO-style) against a Brier-derived reward.
- Evaluated live on ProphetArena and Polymarket against frontier models.
Example datapoint
A sample training example — question, source, and outcome-derived label.
DATASET Market outcomes
RL
Question
Will the US Fed cut rates by more than 25bps before end of Q3?
Question source
Wall Street Journal Jun 12, 2025
Fed officials signal openness to larger cuts if labor market softens
Label
No.
Type
binary
Confidence
Label source
Federal Reserve Sep 30, 2025
FOMC statement: 25 basis point reduction in target range
Results
Benchmark comparisons against frontier models.
ProphetArena Overall Leaderboard
Foresight V3 holds the #1 spot on ProphetArena's live benchmark, ahead of Gemini 3 Pro and GPT-5.2 — while being 10–100× smaller than the frontier models it beats.
Live Polymarket Benchmark
On 251 live Polymarket questions, Foresight-32B achieved Brier score 0.199 vs. GPT-5's 0.207 — with 69% lower calibration error (ECE 6.0% vs. 16.1%) and positive simulated trading profit while frontier models lost money.
Read more
Papers, models, datasets, notebooks, and write-ups for this case study.
Blog post blog.lightningrod.ai
How we built the top-ranked AI forecaster
Leading live prediction-market benchmarks against much larger general models
Blog post blog.lightningrod.ai
Smaller models, stronger results on live markets
Sharper calibration and profitable live trading without frontier-scale compute
Blog post blog.lightningrod.ai
Training on what actually happened
Turning time-stamped history into supervision your leadership team can trust
Paper arxiv.org
Supervision from real-world outcomes
Methodology for replacing manual labels with resolved events
Paper arxiv.org
RL for sharper probability forecasts
How outcome-based training improves calibration and accuracy
Paper arxiv.org
Self-improving forecasting models
Self-training loops that lift prediction quality without new human labels
Notebook github.com
Polymarket backtesting
Evaluate LLM forecasting performance against historically resolved Polymarket prediction markets
Model huggingface.co
Foresight research model
Large model tuned with reinforcement learning for market-style questions
Dataset huggingface.co
Market questions with resolved outcomes
Pairs of historical context and later ground truth for forecasting research
Dataset huggingface.co
AI futures 2025 scenarios
Forward-looking AI questions with timestamps for backtesting forecasters