Building the future workforce of the AI economy 🤝
United States
- Tweets 3,156
- Following 351
- Followers 12,050
- Likes 3,884
Pinned Tweet
Introducing Handshake AI—the most ambitious chapter in our story. We leverage the scale of the largest early career network to source, train, and manage domain experts who test and challenge frontier models to failure for the top AI labs.
44
23
214
359,242
Exciting to see @joinHandshake alongside some of the teams advancing the next generation of AI.
Proud to be part of @nvidia's Nemotron ecosystem!
3
4
17
1,401
ChatGPT for Education analyzed thousands of professional chats with AI. After removing routine tasks like email and calendaring, a little over 40% of what remained was people asking questions and doing work outside their role.
Roles are blurring. That makes skills-based and project-based hiring more important than ever: people are stretching beyond their job descriptions, and employers increasingly want that flexibility too.
1
2
8
827
Handshake works with thousands of working scientists to measure where frontier AI falls short on real research tasks, and to build the data that closes the gap.
We tested Gemini 4, Muse Wonder, GPT-6.1 Sol, and Claude Opus/Sonnet 5.5 on images scientists interpret at work.
None beats GPT-6 Astra, which fails on 50% of the 161 images.
How can lab-in-the-loop AI cure disease if models can't interpret the routine experimental results?
3
690
Congratulations to @reflection_ai on Beam, released today with a full safety report.
Handshake is named in the report as a source for the conversations Beam was tested against, and among the partners who tested the model before release.
Safety testing is expert work. Before a model ships it has to face the requests that matter: self-harm, child safety, cyber offense, fraud. The hard cases are not the obvious ones. A nurse or a researcher can ask something that sounds dangerous and isn’t. Someone has to write that case and decide where the safe answer ends.
Our people wrote those cases for Beam and judged every turn. Making AI safe takes the same expert judgment as making it useful, and we do both.
Report: reflection.ai/beam-safety-re…
Beam Safety Technical Report — Reflection
Make intelligence open and accessible to all
reflection.ai
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
1
15
2,037
We collaborated with @NVIDIA AI to post-train Nemotron 3.5 Super VL on over 2,000 data points from experts on top enterprise R&D teams, doubling its accuracy on life sciences artifact interpretation.
@joinHandshake is building accurate, efficient models to let researchers accelerate their work and focus on the scientific decisions that drive discovery.
Read the research report: joinhandshake.com/research/a…
5
7
45
6,025
Fellows: your dashboard is changing so you can see what you earned, how it was calculated, and when it lands. You’ll see base pay, incentives, and adjustments by project, plus a reason for every change and payment status.
Rolling out across projects over the next few weeks.
Full update here: joinhandshake.com/blog/our-t…
1
9
2,884
What a week at @COLM_conf in SF! We loved meeting the research community, hearing about the work people are doing, and sharing more about our ATLAS benchmark collection at the Handshake Al booth.
Also, a huge thank you to our friends at @Cerebras for cohosting Cafe Compute with us. It was a blast!
Explore the full set of ATLAS benchmarks → joinhandshake.com/research/b…
1
2
17
1,177
“The model’s capabilities are pretty spiky.” — Jonas Mueller
The ultimate benchmark for new models is how useful professionals find them to get stuff done. Explaining benchmaxxing on The Information TV this morning, and why there’s no substitute for actual feedback from scientists/doctors using the model to solve their problems.
2
1
9
942
The best GRE prep used to be a budget question. This benchmark says it doesn't have to be.
@joinHandshake ran a controlled study with 2800+ students comparing the impact of AI with human tutor. Two interesting findings: (1) AI and human produced similar learning gains, and (2) AI is 70x (Opus) to ~900x (Gemma) cheaper. See: ATLAS-Education joinhandshake.com/research/b…
1
3
12
1,402
Agent benchmarks should be grounded in how real people actually interact. This often requires user simulation, but current models often lack fidelity.
After months of development, our research team has published CUE, a simulator that outperforms state-of-the-art approaches on fidelity and its ability to reproduce outcomes with real users.
We’ve released the research, code, and trained embedding model openly for researchers to build better multi-turn benchmarks!
⚠️ User simulators are increasingly used to evaluate AI agents in multi-turn interaction. But we find that looking like real users ≠ reproducing real evaluation outcomes.
📢 We introduce Calibrated User Embeddings (CUE) to bridge this gap across LLM simulators.
🧵 1/8
9
1,136
Handshake retweeted
A user simulator isn’t useful just because it looks like a real user. AI agents should also succeed/fail with simulated users the same way they do with real ones.
To fix this, we open-sourced CUE, a new simulator to help you build more ecologically valid multi-turn benchmarks.
⚠️ User simulators are increasingly used to evaluate AI agents in multi-turn interaction. But we find that looking like real users ≠ reproducing real evaluation outcomes.
📢 We introduce Calibrated User Embeddings (CUE) to bridge this gap across LLM simulators.
🧵 1/8
3
3
37
2,176
As many industries adopt AI, we sat down with Douglas Melsheimer, Co-Head of Emerging Tech Investment Banking at @jpmorgan, to discuss how it’s changing the way we work.
1
2
13
1,380
