Your dashboard said OK. Your agent crashed.
This weekend
@neatlogs' daily email told me 6.2% of my test runs had failed. I checked all 81: 9 had crashed, 11.1%. Four crashes were recorded as SUCCESS. (Test runs I crashed on purpose, with the tracing tools as anyone downloads them.) The email added up what it was given. The tracing code that writes the record got it wrong, and nobody checks the checker.
So I built Trueup.
Trueup Witness is a second sensor for your AI agent, in two lines of code. It watches what really happened on the connection and compares it with the neatlogs record. When they disagree, it writes a red TRACE DISAGREE step into that same record. Then Trueup's agent finds the cause in the tracing code, writes the fix with tests, proves it and sends it upstream.
neatlogs:
Live, with real Claude calls through the neatlogs model key: I cut the connection mid-reply and crashed an agent run. Before my fixes, neatlogs recorded SUCCESS. After, ERROR. 2 wrong records to 0, and no false alarms in 40 normal calls.
I also traced Trueup's own agent in neatlogs, and its alert rules caught my mistakes: 31 replies the AI sent in the wrong format, 27 false alarms from a bug in my own comparison, and 13 failed fix attempts. Each was fixed and kept on record.
@EntireHQ:
Trueup was built in recorded Entire sessions, and each fix links to how it was made. The session where the fixer wrote and proved its first fixes:
entire.io/gh/Jennycruzy/true…
It uses Entire's code graph to find the faulty code and check what a fix could break, and runs as an Entire command: entire trueup.
cfo.ai (
@arithecfo):
Ari built the business plan and a financial model with me, on costs I really measured: a Witness check costs $0, a fix about $0.52. It got one number wrong, I corrected it, it re-ran the plan. In the forecast, at $199 a team 94 paying teams break even, while $99 runs out of cash. Rule from the slow-growth case: pilots first, hire later.
cfo.ai/s/kVKIburWzn09Baoy
Fixes sent upstream:
To Arize, Traceloop and neatlogs. Arize's head of open source reviewed mine: fixing it is "the right goal". neatlogs runs that same Arize code for agents, so one fix repairs both.
github.com/Arize-ai/openinfe…
github.com/neatlogs/neatlogs…
github.com/traceloop/openllm…
Built in public all weekend:
The thread Arize's maintainer replied to:
x.com/jennyoliver57/status/2…
12 Oct update:
x.com/jennyoliver57/status/2…
11 Oct update:
x.com/jennyoliver57/status/2…
Crash it yourself:
trueup.38-49-216-59.sslip.io…
Code and every number:
github.com/Jennycruzy/trueup
@neatlogs @EntireHQ @arithecfo @akshay_hangloo #neatHack