Guardrails, evals & simulation for AI agents. Bring your agent to real-world level.

Pinned Tweet
We're launching vibe training. Describe what your agent should and shouldn't do. We generate the edge cases, build the test set, train a model calibrated to your policies. In minutes.
3
2
19
437,291
Plurai retweeted
כמה גאווה! החבר'ה של Plurai בשיתוף פעולה מדהים עם נביוס ואנבידיה בו הצליחו להוריד את העלות של מודלי AI פי 44 בזכות היכולת שלהם לבנות מודלי שפה קטנים למשימות ספציפיות. זה העתיד ועכשיו הוא כבר ההווה.
Excited to announce our collaboration with @nvidia and @nebiusai at NVIDIA #gtcberlin! We post-trained NVIDIA’s Nemotron 3.5 Lightning into a specialized model that outperforms Claude Haiku 5.5 at 44× lower cost and matches OpenAI’s GPT-6 Luna at 40× lower cost. With Plurai, any engineer can go from a task description and a few production examples to a post-trained, benchmarked, production-ready model in under 20 minutes. Works on any open-source model. Full research: plurai.ai/blog/vibe-post-tra…
10
4
199
18,917
Plurai retweeted
Excited to announce our collaboration with @nvidia and @nebiusai at @nvidia GTC Berlin! We post-trained @nvidia 's Nemotron 3.5 Lightning into a specialized model that outperforms @AnthropicAI Claude Haiku 5.5 at 44× lower cost and matches @OpenAI 's GPT-6 Luna at 40× lower cost on a real-world enterprise task, with @nebiusai Token Factory providing the infrastructure to scale. With @pluraiAI 's vibe-training, any engineer can go from a task description and a few production examples to a post-trained, benchmarked, production-ready model in under 20 minutes. What excites me most is that this same workflow can be applied to any open-source model, turning it into a specialized, frontier-beating model at a fraction of the cost. Huge thanks to our incredible partners at @nvidia and @Nebius for the close collaboration and for pushing the boundaries of what's possible together. This is just the beginning, with much more to come! Full research: plurai.ai/blog/vibe-post-tra…
3
6
31
29,955
Plurai retweeted
Excited to announce our collaboration with @nvidia and @nebiusai at NVIDIA #gtcberlin! We post-trained NVIDIA’s Nemotron 3.5 Lightning into a specialized model that outperforms Claude Haiku 5.5 at 44× lower cost and matches OpenAI’s GPT-6 Luna at 40× lower cost. With Plurai, any engineer can go from a task description and a few production examples to a post-trained, benchmarked, production-ready model in under 20 minutes. Works on any open-source model. Full research: plurai.ai/blog/vibe-post-tra…
3
4
46
13,393
Plurai retweeted
You can’t safely test an enterprise agent on a real company’s data. So we built a company for it to work in. Era is live today, and it’s free. It generates a complete simulated enterprise that behaves like a real one across Salesforce, Slack, Jira, Zendesk, Gong, Deel and more, along with cloud databases and storage. Agents interact with it through live MCP and API interfaces. People leave. Deals change. Records get duplicated. Permissions differ across systems. And because Era generated the company, it knows the exact ground truth. Test, benchmark and improve agents against realistic enterprise workloads, use the failures for targeted post-training, then rerun the same environment to measure the impact. Huge thanks to our research partners @NVIDIA, @Decart, @Composio, @openlayerco, @Deel, @Eragon and @Plurai, with more coming soon.
173
119
837
582,754
Plurai retweeted
Second @Gartner_inc mention for Plurai. This time it's "8 Emerging Practices for Testing AI Agents" - citing Plurai #Evals & #Guardrails as an example of building purpose-built small models for scoring, completeness, and safety checks. Testing can't stop before launch. And you can't grade every production interaction with a frontier model. gartner.com/document-reader/…
2
3
142
If you use LLM-as-judge, this one is for you. (bookmark it) Most teams validate their agent's outputs by calling a frontier model as the judge. It works, until it doesn't. Three problems stack up fast: → Cost: you're hitting a frontier API on every turn, every tool call, every response. In production that burns millions. → Latency: bigger models, remote calls, slow reasoning on every check. → Blind spots: frontier models don't actually know your domain. In finance, insurance, or healthcare, they miss the keywords and principles your work depends on. So I walk through a different approach: train your own small LLM judge. Instead of a giant model, you start with a small one and let the system generate the training data for you. It decomposes your domain, samples synthetic examples, runs them through a debate arena where judges reach consensus, then trains on the refined set. The result is a judge that's cheaper, faster, and more accurate on your data than Gemini, Claude, or GPT, with an OpenAI-compatible endpoint you can even deploy on-prem. I show the whole thing end to end, using a Claude Code plugin and a web interface, with a real insurance RAG grounding evaluator as the example. You can get the plugin here: github.com/plurai-ai/plurai-… Here's the full breakdown: 00:00 - Intro 00:12 - Three problems with using frontier LLMs as judges 01:05 - A different approach: train your own small judge 01:31 - How it works (synthetic data and a debate arena) 02:50 - Installing the Claude Code plugin 04:03 - Defining your task with /eval 04:34 - Example: an insurance RAG grounding evaluator 05:51 - Kicking it off and giving early feedback 06:26 - Choosing labels, domain, and strictness 08:30 - The web interface and dashboard 09:52 - Bringing your own example data (optional) 10:26 - The finished model: endpoint, accuracy, and speed 11:16 - Control, on-prem deployment, and interpretability 11:57 - Benchmarks vs frontier models and the GitHub repo 12:30 - Outro I worked with the @pluraiAI team on this. Thanks for sponsoring the video.
27
57
355
30,671
Plurai retweeted
We're launching vibe training. Describe what your agent should and shouldn't do. We generate the edge cases, build the test set, train a model calibrated to your policies. In minutes.
3
2
19
437,291
Plurai retweeted
Air Canada’s chatbot once literally made up its own refund policy in court and won a lawsuit for the customer, not the airline. There’s a new term being coined right now called vibe training by the company @pluraiAI, and they’ve basically built a way to use tiny, fast models as guardrails to catch hallucinations in sub-100ms and the cost is over 8x lower than GPT-5-mini. 🔥👉 They’re live on Product Hunt today: producthunt.com/products/plu… If you’re building agents, go check them out, grab the free trial, and show them some love on the launch! 🫶 The best part? You don’t need a PhD in AI. Sponsored by Plurai.
Made with AI
Paid partnership (ad)
4
83
11,992
Plurai retweeted
This team just coined the concept of vibe training. Build real-time, tailored evals and guardrails for your agent, with high accuracy at a fraction of the LLM cost. Launching today on @ProductHunt.
Yesterday blew past every expectation. I barely slept (2 hours, if I’m honest)… and now we’re heading straight into our #ProductHunt launch and I need you! 🚀 Because something clicked. We launched vibe training - and within hours, thousands of agent builders started creating evals and guardrails for their own use cases! It’s moving fast. Because the truth is simple: Building agents is easy. Making them reliable in production is not. That’s what vibe training fixes. If you’ve been following, building with us, or just rooting from the sidelines — we need your support ❤️ • Open the link • Hit upvote • Drop a quick comment This takes 30 seconds and directly impacts our ranking. Let’s push this to the top today producthunt.com/products/plu…
Paid partnership (ad)
3
1
4
329
Plurai retweeted
Within hours of our launch, thousands of agent builders were already live with vibe training. Today we're on Product Hunt. An upvote takes 30 seconds. 👉 producthunt.com/products/plu…
Yesterday blew past every expectation. I barely slept (2 hours, if I’m honest)… and now we’re heading straight into our #ProductHunt launch and I need you! 🚀 Because something clicked. We launched vibe training - and within hours, thousands of agent builders started creating evals and guardrails for their own use cases! It’s moving fast. Because the truth is simple: Building agents is easy. Making them reliable in production is not. That’s what vibe training fixes. If you’ve been following, building with us, or just rooting from the sidelines — we need your support ❤️ • Open the link • Hit upvote • Drop a quick comment This takes 30 seconds and directly impacts our ranking. Let’s push this to the top today producthunt.com/products/plu…
8
4
15
1,735
Plurai retweeted
Yesterday blew past every expectation. I barely slept (2 hours, if I’m honest)… and now we’re heading straight into our #ProductHunt launch and I need you! 🚀 Because something clicked. We launched vibe training - and within hours, thousands of agent builders started creating evals and guardrails for their own use cases! It’s moving fast. Because the truth is simple: Building agents is easy. Making them reliable in production is not. That’s what vibe training fixes. If you’ve been following, building with us, or just rooting from the sidelines — we need your support ❤️ • Open the link • Hit upvote • Drop a quick comment This takes 30 seconds and directly impacts our ranking. Let’s push this to the top today producthunt.com/products/plu…
4
3
16
2,738
Plurai retweeted
Today we're launching vibe training. Describe what your agent should and shouldn't do. We generate the edge cases, build the test set, train a model calibrated to your policies. In minutes. Start Vibe-training: plurai.ai/launch
Big day for us, finally sharing what we’ve been cooking for a while. Over the past year, we kept seeing the same pattern: AI agents look great in demos, until real users break them. Today, we’re fixing that with 𝘃𝗶𝗯𝗲-𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 to build real-time, tailored evals and guardrails for your agents, in minutes. Define your intent with a prompt or a few examples. We generate edge-case datasets, and train a model aligned to your use case, outperforming state-of-the-art LLMs at a fraction of the cost. (Research paper with benchmarks in the comments) If you’re building AI agents, don’t let your users be the ones who discover the failures. Be the one who makes AI agents reliable in production and takes control at scale. Start vibe-training for free: plurai.ai/launch
4
9
220
Vibe train your AI agents. This new method can replace LLM-as-a-judge for production agents. Most teams point a giant LLM at their agent's output and call it evaluation. It works, but it comes with two real costs: - It's slow and expensive at inference time - It misses the domain-specific failures that actually matter to your use case Vibe training flips the whole setup. Researchers at Plurai distill a small language model that's specialized for your agent's exact behavior, your edge cases, and your failure modes. The SLM becomes your evaluator and your runtime guardrail in one. Here's why this is a big deal: - Cheap enough to run inline on every agent step, not just offline batches - Catches the failures that generic LLM judges shrug off - Same model guards production and grades it, so eval and runtime stay in sync A small specialized model beating a giant general one is becoming a pattern. Distillation is quietly turning into one of the most underrated techniques for shipping reliable agents. Try it here: plurai.ai/launch Paper: plurai.ai/papers
Big day for us, finally sharing what we’ve been cooking for a while. Over the past year, we kept seeing the same pattern: AI agents look great in demos, until real users break them. Today, we’re fixing that with 𝘃𝗶𝗯𝗲-𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 to build real-time, tailored evals and guardrails for your agents, in minutes. Define your intent with a prompt or a few examples. We generate edge-case datasets, and train a model aligned to your use case, outperforming state-of-the-art LLMs at a fraction of the cost. (Research paper with benchmarks in the comments) If you’re building AI agents, don’t let your users be the ones who discover the failures. Be the one who makes AI agents reliable in production and takes control at scale. Start vibe-training for free: plurai.ai/launch
5
9
59
5,795
Vibe train your AI agents. There's a new method that could replace LLM-as-a-judge for production agents. Most teams rely on a giant LLM as a judge to evaluate and guard their agent. But it has two major drawbacks: - It's slow and expensive at inference time - It often misses domain-specific failures Vibe training flips this. Researchers at Plurai distill a small language model that's specialized for your agent's exact use case. The SLM becomes your evaluator and your runtime guardrail, both in one. The training data isn't hand-curated either. They spin up a swarm of adversarial agents that debate and stress-test every use case your agent is supposed to handle. That synthetic interaction data trains the specialized SLM. So the judge actually understands what "wrong" looks like in your specific domain. The reported gains vs. standard LLM-as-a-judge setups: - ~8x faster inference - ~50% fewer evaluation errors Smaller, faster, and more accurate because it's specialized for the job. The SLM-for-agents thesis is playing out in a very concrete way. If LLM-as-a-judge is your current evaluation layer, this is worth benchmarking against. Paper link in the replies.
20
25
153
11,666
Plurai retweeted
THIS IS REVOLUTIONARY Building agents from now on is going to be a totally different thing. much more reliable much faster much cheaper and as far as I know, it is currently free. they train a small language model in a super sophisticated way that makes unlabeled data labeled, using agents that debate to get the best label. it is brilliant...
Big day for us, finally sharing what we’ve been cooking for a while. Over the past year, we kept seeing the same pattern: AI agents look great in demos, until real users break them. Today, we’re fixing that with 𝘃𝗶𝗯𝗲-𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 to build real-time, tailored evals and guardrails for your agents, in minutes. Define your intent with a prompt or a few examples. We generate edge-case datasets, and train a model aligned to your use case, outperforming state-of-the-art LLMs at a fraction of the cost. (Research paper with benchmarks in the comments) If you’re building AI agents, don’t let your users be the ones who discover the failures. Be the one who makes AI agents reliable in production and takes control at scale. Start vibe-training for free: plurai.ai/launch
1
3
9
343
I used to pay for the most expensive AI models just to double-check my own agents. It felt like a "safety tax" I had to pay, but it was killing my margins and making everything feel slow. I was basically paying twice for the same result. Plurai finally fixed this. Instead of a giant model, you train a tiny one that only cares about your specific rules. You just type what you want in plain English, and it builds a custom safety net in minutes. It runs instantly and costs almost nothing. This is how you actually move from a prototype to something that works at scale. Check it out:
Big day for us, finally sharing what we’ve been cooking for a while. Over the past year, we kept seeing the same pattern: AI agents look great in demos, until real users break them. Today, we’re fixing that with 𝘃𝗶𝗯𝗲-𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 to build real-time, tailored evals and guardrails for your agents, in minutes. Define your intent with a prompt or a few examples. We generate edge-case datasets, and train a model aligned to your use case, outperforming state-of-the-art LLMs at a fraction of the cost. (Research paper with benchmarks in the comments) If you’re building AI agents, don’t let your users be the ones who discover the failures. Be the one who makes AI agents reliable in production and takes control at scale. Start vibe-training for free: plurai.ai/launch
Paid partnership (ad)
15
31
99
38,868
Plurai retweeted
I've made a ton of money helping companies implement LLM-as-a-judge evaluations. LLM Judges provide a ton of value. But the hard part is choosing the model to implement the judge. • The family of GPT-5 models is very good, but slow and expensive. • Models like Gemma and Phi are fast and cheap, but not that good. Most of the time, you can only run a percentage of your traffic through the model (otherwise it would be too expensive and slow). But now, there's a better strategy.
16
24
213
30,069