What we did
- Ingested supplier performance metrics, ERP shipment logs, and industry news with timestamps.
- Generated forward-looking disruption questions anchored at each point in history.
- Labeled each question using the actual downstream outcome (ASN slip, dwell spike, price move).
- Trained a 32B model with RL to output calibrated probabilities, then benchmarked against GPT-5.
Example datapoint
A sample training example — question, source, and outcome-derived label.
DATASET Disruption outlook
RL
Question
Will Acme Motors Tier-1 supplier FlexAlloy (plant Shenzhen) log a >3-day ASN slip on part HV-220 busbars in the next 30 days?
Question source
Supplier portal Jan 8, 2026
On-time performance slipped to 94% vs. 98% prior quarter
Label
Yes.
Type
binary
Confidence
Label source
ERP shipment log Feb 4, 2026
ASN delayed 11 days — weather and carrier capacity cited
Results
Benchmark comparisons against frontier models.
Aggregate Performance vs. GPT-5 and gpt-oss-120b
The trained model beats GPT-5 and gpt-oss-120b on every headline metric: Brier 0.0791 vs. 0.1203 (GPT-5), Brier Skill Score +16.9% vs. −26.4%, and ECE 0.0525 vs. 0.1740 for the pretrained base — a ~70% reduction in calibration error.
Calibration vs. GPT-5 on Supply Chain Predictions
The trained model closely tracks the perfect-calibration diagonal — when it says 30% probability, roughly 30% of disruptions materialize. GPT-5 and the base model severely under-predict high-risk events. Precision@10% improves 4× (35% vs. 9%).
Read more
Papers, models, datasets, notebooks, and write-ups for this case study.
Blog post blog.lightningrod.ai
Forecasting supply chain disruptions
End-to-end walkthrough of training a supply chain disruption forecaster
Paper arxiv.org
Foresight learning for supply chains
Method paper on outcome-based training for supply-chain forecasting
Dataset huggingface.co
Supply-chain predictions
Timestamped supply-chain questions with resolved outcomes