nitya's notes + building a search engine @exaailabs

San Francisco, CA
this is incredible
Our newest book is finally here! Making the Modern Laboratory is a spectacular 504-page hardcover that tells the story of everything in the research laboratory and how it came to be, from pipettes to heat-resistant glass to microscopes and more. It features dozens of articles (most of which will never appear in print), 100+ full-color images, and original writing from 36 authors. It's printed on Arena Natural Rough paper from Fedrigoni. These books are THICC and would make a great gift for both the science-curious and the working scientist. :) We have spent the last year working on this project. It has been an insane labor of love. The best way to support our independent science journalism is to buy a copy. <3 Pre-order for $49, with free shipping: press.asimov.com/articles/mo…
16
1,068
signing up for a race is a competitive sport now someone give me your Copenhagen half bib pls
3
5
1,427
TIL the onboarding for macOS 9 had a whole section explaining what the internet is and how it works
41
166
3,936
128,843
a lot of erratic behavior by smart people on twitter can be explained by studying scientific heretics often when I am scrolling this app I think about Brian Josephson, who at 22 proved that electric currents can flow between two superconductors if separated by a thin layer of insulating material. He was early to something that was correct and rewarded the highest prize for it - Nobel for physics. He then spent the rest of his career studying things like telepathy, refusing to change what he worked on. Being early and right, makes it hard to believe people when they tell you you're wrong later on.
6
1,111
Product team at Exa is hiring! come work with me the work is just really interesting - building time versioned search of the web, productionizing research across our retrieval team, driving search quality, API design and there is still so much more to do
Exa just passed $100m in annualized revenue. We've grown almost exactly 10x per year for 3 years. We now handle 2 billion requests a month from 200k+ teams. And it’s still so early: We predict agents will search 1000x more than humans by 2030. Agents don't search like humans. They write long, complex queries and want all the world's data, not just the web. We've spent 5 years building a search engine designed for them. We now serve personal agents, AI labs, coding agents, GTM agents, really any agent. Exa gives these agents access to 100s of billions of web documents and tons of proprietary data. Our customers are increasingly noticing that information is the bottleneck to agent performance. While the world seems close to perfect intelligence with models like Fable and Astra, we are very far from perfect information. To our small but legendary team, thank you for getting us here, from engineering to research to GTM to ops. To our customers, it is an honor to serve you. We have some exciting releases coming soon and can't wait to share them 🫡
19
5
280
16,797
Nitya retweeted
Sf summer is here
4
2
37
884
up the hill from stinson beach
6
38
1,389
evals are also distinct across tasks in ways you might not expect retrieval beyond parametric memory requires specific focus on recency, tasks that are consistently changing - it is an important part of how I think about product quality. Wrote about this + open sourced our code evals a while back, linked below
Evals have been coming up more and more in my conversations with podcast guests and PM friends. Nearly half of the 25 awesome PM job openings I shared last week ask for experience writing evals. And leading companies keep sharing what investing in evals bought them: — @tryramp took its automatic receipt collection from 35% to 83% accuracy. — @Shopify shipped an AI workflow builder that's 2.2x faster and 68% cheaper than the frontier-model setup it replaced. — @harvey__ai rebuilt its AI contract reviewer, nearly doubling its internal quality score. — @cursor_ai tuned its Auto Balance routing, with much higher user satisfaction at 41% lower cost. So I asked the 🐐s of evals, @HamelHusain and @sh_reya, to write an advanced sequel to their very popular "Building eval systems that improve your AI product." Drawing on their work with 50+ AI companies, they share the key step most teams skip, what you should (and shouldn't) automate, and a free plugin that lets a coding agent do most of the heavy lifting. Read it here: lennysnewsletter.com/p/advan…
2
21
1,287
me: how do i open pdf claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.
231
2,014
30,466
765,354
btw walter benjamin was thinking abt this in 1935 <3 u children think u came up with aura
i think you are accidentally rediscovering walter benjamin’s “aura” problem. he basically asks if information/reproduction can give you the object more efficiently, why does being physically in the presence of the real thing still matter museum.stanford.edu/exhibiti…
5
1
19
1,821
live demos are really hard, remember pitching a new product in front of 100 people once and it was unexpectedly laggy I know it is openai so we should have a high bar, but I would take this over a pre-recorded demo most days
HOLY SHIT OpenAI's live demo for "Dots" completely FLOPPED💀 and then they CUT THE AUDIO on the official stream to hide it I caught the original:
2
34
5,704
I remember my demo was right after @MelkeyDev's who had beautiful formatted slides god damn it
1
1
238
Powered by Exa snapshot! To eval forecasting models, aaru used Snapshot to give their models access to the web at a specific time in the past. We're seeing people use it for prediction models, backtesting, preventing web leakage during RL runs
Simulation has the potential to be transformative, but only if it's accurate. Today, we're sharing results of a comprehensive evaluation across 2,993 questions, with an industry-leading mean TVD of 7.62% and an MAE of 3.53%. More below, including the full post on our site.
1
17
911
one day nitya you will help people predict the future by creating a time machine of the web I love my job
3
117
Exa snapshot! Search back in time
I used Shortcut to cut EPS forecast errors by 20% vs. Wall Street consensus. I did this by backtesting Q2 2026 earnings forecasts for 113 small caps covered by 3+ analysts on Shortcut with Sol medium effort. Even after raising consensus by 2.7%, the best simple adjustment I found in hindsight, it only cut median errors by 4%, versus Shortcut’s 20%. The reason I’m exploring this is because I believe that agents will give investors a starting point for companies they might not have previously had time to research. I used Shortcut’s generated spreadsheet models from the backtest to inspect, beyond just the final EPS estimate, which approaches held up. In 40% of forecasts, agents built up to EPS from revenue, costs, taxes, and share count. For that group, median forecast errors were 32% lower than the Street’s. I also dug around to see how others are approaching earnings prediction. LSEG’s StarMine improves on consensus by giving more weight to recent estimates and historically accurate analysts. A 2024 paper I found reported that GPT-4 outperformed analysts at predicting whether annual earnings would rise or fall, but that was later withdrawn after a co-author found inconsistencies in the data and analysis during replication. That made how I test just as important as the forecasts. So I realized I needed to limit the agent to information available before each earnings report and account for what it might already know from training. I chose Q2 reports to reduce the in training-data risk since Sol was trained on data up to mid February. For web search, I restricted access to information from before each report and used TypeSafe’s Jev to screen pages for leaked results, which I found more reliable than Haiku in my tests. I’m now moving to Exa Snapshot to retrieve pages as Exa stored them at the cutoff. These controls help, but forecasting before results exist is the real test. October earnings season seems like a great time to expand and publish forecasts. This is still an early experiment, and I’d love to hear what y’all think I should try next and whether this could be useful :)
2
12
2,159
Was running through a cloud when the sun came out on this trail I’ll never forget it
Bay Area trail runs are undefeated
37
2,482
search through time
We built a time machine for the web. Introducing Exa Snapshot: an index of 400 billion historical snapshots of webpages that lets you search as if it's the past. Snapshot is already being used for backtesting prediction models, RL at labs, exploring the pre-AI web, and more.
21
1,344
by far the coolest product I’ve worked on at Exa so far - versioned view of the web This has taken a ton of work and it is truly just insane to see this in the wild
We built a time machine for the web. Introducing Exa Snapshot: an index of 400 billion historical snapshots of webpages that lets you search as if it's the past. Snapshot is already being used for backtesting prediction models, RL at labs, exploring the pre-AI web, and more.
4
64
4,801
We are on the cusp of a wave of new therapies for some of the worst diseases. But the world won’t benefit unless the US fixes its drug regulatory system. My new essay for @nytimes, on how slow clinical trials are now the biggest obstacle to curing cancer. nytimes.com/2026/09/04/opini… - I interviewed dozens of researchers, especially oncologists at leading U.S. centers. A striking consensus emerged: science is no longer the main bottleneck to new cancer drugs. It is our ability to test discoveries in patients through clinical trials. - The cost of starting a Phase 1 trial in America has roughly doubled over the past decade. As a result, companies increasingly take early trials abroad: Australia’s Phase 1 trial volume has nearly doubled in a decade, driven mainly by U.S. companies. - Unfortunately, the underlying incentives are badly asymmetric: Institutions can be blamed for harms caused by moving too fast, but almost no one is blamed when patients deteriorate during avoidable delays. One doctor called the emerging system “ritualized safety over actual risk assessment.” Or as, @DavidHongMD put it: "We often forget that the biggest risk is the cancer itself." - This problem is becoming more urgent because medicine itself is changing. Sequencing, biological engineering and A.I. make increasingly personalized therapies possible. But our regulatory system was mostly built for standardized drugs tested in large populations. - @sytse, the co-founder of GitLab, shows what personalized medicine can achieve: after relapsed osteosarcoma and being told there were no options left, he pursued a highly individualized approach and has now been cancer-free for a year. But doing so required extraordinary resources and regulatory expertise. - Pierce Ogden’s father was less lucky. After molecular analysis identified a drug that might target his glioblastoma, the manufacturer agreed to provide it. But administrative barriers delayed access until it was too late. “My dad was ready to try anything,” Pierce told me. “But the system is paternalistic.” - The A.I. revolution is making this bottleneck more important, not less. A.I. relies on relevant data. Information from early-stage trials could compound with A.I. tools to achieve truly revolutionary medicines. Without the data, this is far less likely to happen. - Another important shift is that innovation increasingly comes from academic labs and small biotech companies rather than Big Pharma. These small companies find it far harder to unable to absorb delays and regulatory barriers. - Apart from cancer, China is the biggest winner from America's outdated medical regulations. China has a much faster trial system, with testing often starting a full year earlier. This allows Chinese pharmaceutical companies to experiment and improve medicines while American companies play with mice. Today, half of all drugs licensed by major pharmaceutical companies originate there, up from less than 5 percent only a decade ago. - But we don't need to copy China. The best model is Australia: lots of on-site scientific and ethics reviews, and requirements that are proportionate to small, early-stage trials. Phase 1 studies there begin roughly 6–12 months faster, without any notable increases in adverse safety events. - Operation TrialBlazer, a 2026 HHS initiative is a good start in this direction, but we need legislative action by Congress to truly make Phase I trials faster and more efficient! I want to thank everyone who helped me with this article: everyone I interviewed and the amazing editors at the Times. This is the result of a months long journey of extensive interviews and research. Special thanks go to those who came on the record. One of the features of the system is an atmosphere of fear, where practitioners are afraid to publicly come out and explain these issues. So anyone who does is a hero in my book!
81
411
1,603
549,322