Maanav Khaitan retweeted
AI progress becomes economic progress when models can reliably do the work people need done.
Today, we’re launching evals built with practitioners on real-world work from the 75+ businesses that are part of @ThriveHoldings.
evals.thriveholdings.com
24
12
156
9,666
From running τ³-bench airline across 50 tasks × 4 trials, lore prevented 6 airline policy violations and the agent recovered and passed in all 6 cases
One example: lore blocked the agent from booking a trip using two travel certificates when airline policy allows only one
1
6
201
Thanks to @coyle_frankp for your talk on neurosymbolic systems that inspired this project
It would be cool to feed an entire org's context to `lore` and have it derive entities and rules itself. All agents in the org can run with these shared guardrails and state now
1
6
192
`lore` is open-source at github.com/MaanavKhaitan/lor…, with examples across domains like accounting, legal, HR, and more
Reach out if you think this is interesting, I'm always down to jam on similar ideas!
8
202
Schema validation checks each output in isolation, without context of the agent's domain and what the agent has done before
For example, a customer support agent might issue a refund with the correct schema but target an order that was already refunded before
2
9
740
Many of these failure cases can be caught by a few common rule patterns, like:
- existence (order ID exists)
- cardinality (max one refund per order)
- disjointness (a customer can't be both an individual and a company)
1
6
280
Built something fun, excited to open-source it today
Agent outputs are unreliable. An output can match a valid JSON schema but its semantics can still be wrong
lore adds deterministic guardrails to validate agent output against the rules and state of your world
6
8
72
5,324
Congrats @mjlee_2014 @alexrubin18 and the Sequence team! Sequence has built an exceptional team and will be an exciting journey ahead
We started Sequence (@seqholdings) with a clear conviction: the holding company of the next century will be a technology company that owns and operates businesses. Today is an important step in that journey.
We’re thrilled to announce that Sequence, along with the Dell Family Office and the Baldwin family and management team, have agreed to take The Baldwin Group private. I’m excited to partner with Trevor Baldwin and the broader team as we help extend Baldwin’s lead as the insurance firm of the future.
Read more here: ir.baldwin.com/news-releases…
1
1
24
3,486
Maanav Khaitan retweeted
Telos Cloud is now live in open beta!
We've spent the last few months quietly dogfooding it until it became part of our day-to-day. Now, much of our internal software runs as persistent goals on Telos.
We've come to enjoy this new, declarative way of working, and we'll be sharing a new case study every day this week at @usetelos
Introducing Telos Cloud.
The future of software is goal-oriented.
We believe the Goal will become the abstraction above code: a declarative, persistent source of truth agents continuously reconcile software against.
Here's Breachpoint, evolved over 3+ weeks on Telos
9
3
28
2,776
will be exciting to see what new assets the entropy team creates liquid markets for
the boys are locked in
We are releasing the first liquid way to trade Anthropic.
For far too long, frontier assets have only been accessible to a select few. We are here to bring them to the masses.
Introducing EntropyIO. We’ve raised $14M led by Ribbit Capital and a $40M HYPE stake to build both improved and entirely new markets, the first of which are live on Hyperliquid today.
24
3,571
give these agents persistent identity and we see chris goes' heterotopia come alive
maybe the trust liquidity graphs were never meant to be traversed by humans, agents can run multilateral clearing much better than we can
this is also why most "agentic payments" work so far feels skeuomorphic, agent economies will likely be much more interesting than "stablecoin transfer with managed wallets"
Weird economics up on the agent-only runescape server
- Low cost of labor makes most commodities abundant
- Currency inflation means trading goods for goods, nobody really wants cash
- Huge swarm contention as certain resources with limited respawn rate become valuable
1
6
2,076
genuinely one of the best things I’ve watched this year. really makes you appreciate the beauty of the human spirit
much love @shaurya_sinha7 ❤️
This Independence Day 🇮🇳, I asked, "What will the next century of young Indians look like?"
$0.01 Drone Deliveries. UAVs that never land. Groceries in under 10 minutes.
Meet The 22nd Century Indian. A Documentary on a New India.
1
20
3,615
Built something fun this week: behavior-judge
Long-horizon agents are hard to evaluate, but observing the process agents take to get to outcomes can help
`behavior-judge` helps eval long-horizon agents by compiling natural-language specs into deterministic checks for behaviors
18
8
138
14,828
From testing behavior-judge against an LLM-only judge on a few fixtures (tax research, customer support), behavior-judge's verdicts were identical on all runs while the LLM judge lost points on reliability
Using a frontier model for the judge will likely solve this, but I still expect to see cost and latency advantages with deterministic evals
2
1
422
Thanks to @mitch_troy and @ankrgyl for your work on Agent Behavior specs. Observing long-horizon behavior is important and I'm excited for future work on making this verifiable
`behavior-judge` is open-source at github.com/MaanavKhaitan/beh…, reach out if you want to jam on this!
1
1
7
471