HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world. Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research. All frontier models scored under 50% 🧵

Banger paper from Google Cloud AI Research on agents doing professional engineering work. (bookmark it) For professional engineering deliverables, the best agent configuration averages 50.6%, using 27M tokens and 63 minutes per task. They introduce a benchmark called PLE (Physical-Engineering's Last Exam). It has 85 tasks across electrical, civil, structural, and mechanical engineering. Solving a task means producing a billable deliverable, such as a design package a licensed reviewer would sign off on, from a multi-format project corpus with physics and building-code requirements. Some tasks are written end to end by practicing engineers. Others start from an engineer's problem statement, and an adversarial Creator-Solver agent pipeline builds the project environment, grader, and reference deliverables before experts validate them and make them harder. Frontier models were tested under seven public agent harnesses. Most existing engineering benchmarks are textbook QA or toy design problems that leading agents already saturate. arxiv.org/abs/2610.30537 Chat with Paper: academy.dair.ai/papers/ple-p…

Banger paper from Google. It's standard practice to sample several agent rollouts and trust the answers they agree on. This Google paper shows that agreement can hide shared errors, while disagreement often points to the correct alternative. VeriHarness turns the same base model into an agentic verifier with two jobs. One resolves claims where rollouts disagree by checking workspace evidence. The other challenges claims that every rollout agrees on and looks for requirements they all missed. Across five long-horizon benchmarks, it gives the best selection scores among the baselines tested. With evidence-backed revision, it adds 6.2 points over a single rollout with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. The authors also release about 26,000 rollouts. Paper: arxiv.org/abs/2610.00972 Chat with Paper: academy.dair.ai/papers/verih…

🔍 Disagreement Resolver: Traces conflicting claims back to source files and constraints. ⚔️ Consensus Challenger: Actively refutes shared errors and surfaces missed requirements. ⚖️ Adjudication: Selects and delivers the final artifact. 🌱 Self-evolving verification skills.