Today, Trata is excited to share the revised Hedge-Bench: our benchmark tying open-ending reasoning to prediction capabilities.
We constructed 92 real-world environments derived from hedge fund analysts’ reasoning traces. Agents reason from frozen information snapshots, and their forecasts are evaluated against future real-world outcomes. No model scores above 50% (GPT-6 Astra being the top performer at 44.2%). For frontier, state-of-the-art models, these results highlight a persistent and wide gap between producing plausible-sounding reasoning and making accurate, economically valuable forecasts. In a semi-verifiable domain like finance, where outcomes are delayed and reasoning is difficult to verify, intermediate expert reasoning traces are crucial.
We post-trained
@Meta's Muse-Glimmer-30B on 165 environments with per-turn credit assignment using an LLM-as-a-Judge. We observe evidence of generalization: held-out forecasting accuracy at high effort improved by 10.06 percentage points, while out-of-distribution evaluation scores improved by 3.94 points on
@OpenAI's GDPval, 2.29 on
@Surge’s GDP.pdf, 7.07 on
@Harvey’s LAB, and 4.96 on
@Samaya_AI’s FrontierFinance under their respective metrics.