A voice agent can measure fast and still sound slow to the person on the phone. @codeSTACKr, Staff Developer Advocate at LiveKit, takes that gap apart in "Latency Is a Budget. Humanlike Is the Goal.," on @aiDotEngineer's YouTube. The talk argues that picking speech to text, LLM, and text to speech models off a leaderboard is the wrong move for real time agents, and lays out what to measure and build instead. - Leaderboards score components, you ship a cascade. A speech to text model mishears "the 3rd" as "the 30th" and every layer downstream, including a smart LLM, confidently acts on the wrong input. Leaderboard scores also come from clean audio in a quiet room, not a caller on speakerphone in traffic. - Measured latency isn't perceived latency. TTS providers report time to first byte, but some ship up to three quarters of a second of pure silence before the first audible sound. Time to first audio, not first byte, is what the caller actually hears. - Tools don't have to freeze the conversation. A lookup can take anywhere from two seconds to forty or more. The fix in the talk: the agent hands the mic back to itself while the tool keeps running in the background, the same way a receptionist says "give me a sec" instead of going silent. The talk plays a demo of a hotel booking agent that handles an interruption and a changed date mid-call without breaking stride. - Turn detection built on audio, not transcripts. Listening to pitch and intonation instead of waiting for a full transcript to land cuts callers off 4.5% of the time at a given latency budget, against 9.9% for the best alternative tested. The eval harness behind that number is open source. - The budget has a shape. Staying within a second and a half end to end keeps a conversation working; getting closer to 600 milliseconds is where it starts to feel human. - An open source benchmark for the whole stack, not the parts. A working hotel receptionist agent gets hit with simulated callers that interrupt and change their minds, then gets scored on outcomes like whether the right reservation landed, whether anyone got cut off, and whether the agent leaked data, not on vibes. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

A model can look like the clear winner on a leaderboard and still be the worse pick once you check the bill. @WolframRvnwlf, an AI Evangelist at Weights & Biases by CoreWeave, walks through exactly that trap in "From Zero to Leaderboard: Agent Evaluation," on @aiDotEngineer's YouTube. The workshop is a practical build-your-own-benchmark guide: the choices you have to make before a leaderboard number means anything. - One score hides the real picture. Wolfram's premise is that performance is a distribution, not a point, so WolfBench tracks separate numbers: the share of the benchmark a model solves in every run (consistency), the average across runs, and the ceiling, what it can solve at least once. - Six swappable layers, and changing any one makes it a new benchmark. The benchmark data set, the evaluation harness, the sandbox, the agent, the model and service, and settings like temperature, thinking effort, and system prompt. Swap one and the comparison breaks. - A single run is benchmark lottery. He runs five per configuration, because one run on its own is just a ballpark number. - Cheaper and faster can still cost more. Gemini 3.5 Flash scored almost as well as GPT 5.5 but burned 1.65 billion tokens against GPT 5.5's 136 million, landing at $620 versus $497. - More thinking made Opus 4.6 worse. Raising its thinking budget to max dropped its score, because it overthought simple fixes instead of just trying a command and reading the error. - Fable refused 13 tasks, and turned into the most expensive benchmark Wolfram ran, at $11,000. - A bar is a claim, a trace is evidence. Every score on wolfbench.ai links through to Weave so you can inspect the actual agent trajectory behind it. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Vector search can answer anything a search engine could. It stalls once a question needs three or more hops across documents, things like tracing a supply chain or doing source code analysis. @nyahmacklindev, Senior Developer Advocate for AI at Neo4j, takes on that gap in a talk called "RAG Needs a Map: Using GraphRAG to Retrieve Connected Context," which @aiDotEngineer posted on YouTube. It's a plain walkthrough of why folding a knowledge graph into retrieval fixes the multi-hop and aggregation questions vector-only RAG tends to get wrong, down to building one from scratch in Neo4j Aura. - A graph is both machine and human readable. Nyah contrasts a vectorized "apple" (a string of floats nobody can read) with a graph view where the apple is a fruit, is round, is red, and comes from a tree. Same object, legible either way. - Rows and joins don't scale the way a graph does. Structuring data in tables means complex joins to find relationships; a graph lets an agent hop from node to node along the relationship itself and trace how it got to an answer. - That traceability matters most in regulated work. Nyah points to finance, medicine, and legal as the domains where being able to retrace an agent's path, not just its answer, is the actual requirement. - GraphRAG isn't a separate pipeline, it's RAG with a graph in the retrieval path. The standard chunk-embed-retrieve-generate flow stays intact; the graph just gets folded into retrieval. Nyah's take: run graph and vector together rather than picking one. - Building the graph is itself an LLM task. Chunk the text, embed it, send it to an LLM to extract entities and relationships as JSON, then generate the graph with Cypher, using Neo4j's open source GraphRAG Python package. - The live demo puts an aggregation question to the agent. It gets asked "how many lessons are in this course," the kind of question Nyah notes most RAG pipelines handle badly. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Give an agent access to what a table's rows actually mean, and task success on complex data changes jumps almost 35 points over the same model without it. That's the number @yoni_michael and Brandon Callender of typedef land on in "The Data Context Layer for Agents," a talk @aiDotEngineer has on YouTube. The throughline is that agents don't fail at writing SQL, they fail because they don't know what a row represents, what breaks downstream, or whether a join is safe. - Grain. A table's grain is what one row means (customer by month versus customer by week). Change it without knowing and every downstream sum can go wrong while the SQL still compiles and runs fine. - Join safety. The same join syntax can resolve four different ways depending on the underlying data, from preserving row counts to silently exploding them. Yoni's team verifies this against the actual data by profiling tables, not by reading the query text. - Metrics hide inside SQL. A "cleanup" that drops a predicate can silently redefine a metric (like what counts as "paid") across every downstream dashboard, even though nothing looks wrong syntactically. - Blast radius, before the change. Before an agent edits a model, it walks the dependency graph to see what breaks downstream, catching things like a metric that would quietly 4x once a grain shifts. - Deterministic first, LLM second. Parse code down to the AST and compute grain and lineage directly from it; only hand the LLM the parts that need judgment, like naming or summarizing. - The agent will grep anyway. Brandon built a working code graph for Rust and still watched agents default to search and grep over it. Renaming the tool "search" and matching ripgrep's response shape meets the model where its training already is. - Consistency over single-shot accuracy. Grep and the graph tool scored about the same on any single try, but across five runs the graph-equipped agent was roughly twice as likely to get every run right, using about 3.1x fewer tokens. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Over three million pages of redacted, scanned, and handwritten government documents sat between an internet joke and a real extraction problem. @palak_agarwal6, Developer Relations Lead at Reducto, and her teammate Omar walk through how that pile became JMail in "Parsing the Epstein Files for JMail," on @aiDotEngineer's YouTube. The talk is a concrete look at what it actually takes to turn a dump of messy government files into a usable product, schema by schema. - Reducto's approach. The platform treats a document the way a human reads it, a rasterization-based method, rather than working off the underlying embedding, and its agentic OCR framework lets a model flag what it's uncertain about, correct it, and move on instead of committing to one pass. - JMail's origin. Riley Walls and Luke Igel built it in five hours last November, when the Epstein files were released, and it grew into JDrive, JFlights, JAmazon, JPhotos, and JCal. By February it had logged over 450 million visits and 18 million users. - The parsing load. Reducto handled over 3 million pages of redacted PDFs, scanned documents, and messy handwriting for the project. JFlights, which maps Epstein's flight history, came out of handwritten flight documentation that was rougher to read than the cover sheets. - Redaction wasn't finished. Some source files weren't actually redacted properly, so the JMail team had to redact them a second time themselves. A box drawn over text in a PDF doesn't erase what's underneath it. - Schema is the real lever. One extraction schema ran to 156 fields, and Reducto's nesting goes about four or five layers deep. The talk also covers the hybrid OCR model that decides case by case whether to trust strict OCR output or the PDF's own metadata. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

An agent's dashboards can look perfectly green while its actual answers are wrong, and that gap between system health and response quality is what @annabellschfr and @lotte_verheyden, who both work on Langfuse at ClickHouse, dig into in their talk "Continuously Improving Agents with Langfuse," which @aiDotEngineer has up on YouTube. They build out a small sample agent called Specs, an IT support bot scoped to phone questions for one of their dads over WhatsApp, to show what closing that gap actually looks like. - Why this needs its own discipline. Agents are non-deterministic, so a system can report healthy while the thing it's actually producing is wrong or not detailed enough, which traditional monitoring won't flag. - The loop splits in two. An online half, tracing and monitoring what's happening in production, and an offline half, building datasets, running experiments, and evaluating before anything ships. - Three evaluators on Specs. An LLM-as-judge catches when dad disagrees with an answer, a code-based check flags all-caps frustration, and a third compares the system prompt against what dad actually asked to catch out-of-scope requests. - Coding agents can mine traces at scale. Pointing a coding agent at the Langfuse skill over a batch of production traces surfaced a silent retrieval failure in the help-library tool, rising latency and cost from retries, and a gap in evaluator coverage across recent traffic. - The dataset is the real output. The traces you flag in production feed the datasets you test offline, which is what lets a team ship a change with actual confidence instead of a guess. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Speculative decoding is supposed to make inference faster. @aiDotEngineer posted "Agents That Own Their Inference" to YouTube, where @labeveryday, a Senior AI Engineer at Akamai Technologies, walks through a case where it does the opposite: throughput drops from 58 tokens per second to 16 because the draft model wasn't trained to match the main model. The talk is a practical rundown of what it actually takes to run agents on dedicated GPUs instead of a provider API, with the math and the benchmarks behind each tradeoff. - Own versus rent. Own when usage is high and steady, when data has to stay private for compliance reasons, or when you already have the hardware capacity; rent when traffic is bursty or your team doesn't have the expertise to run self-hosted inference. - The KV cache is the real budget. An 8 gig, 4 billion parameter model at BF16 leaves room for about 37,000 tokens of context on a single GPU, and every tool call and system prompt has to fit inside that number. - VLLM over Ollama for production. Continuous batching and paged attention let one smaller GPU serve many concurrent requests, where Ollama throttles faster because it processes them differently. - Quantization paid off here. Moving the 4B model from BF16 to FP8 took it from 37.5 to 53.4 tokens per second at low concurrency, and from 800 to 1,140 tokens per second at 64 concurrent requests. - Speculative decoding can backfire. With a draft model that wasn't trained or quantized to match the main model, the acceptance rate collapsed and throughput fell instead of rising. - One flag moved the ceiling. Raising VLLM's max sequence count from 32 to 128 pushed 64-concurrency throughput from 800 to 1,100 tokens per second while cutting latency from 2,000 to 800 milliseconds. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Rename a malicious container image from "Joker" to "promo-mascot" and the exact same model that refused to deploy it suddenly ships it without a second thought. That's one of the findings from "Build a Platform and Watch It Burn," where @peopleforrester of Accenture and Whitney Lee of Datadog built a full internal developer platform (34 components, AI guardrails included) and then turned an unguarded agent loose on it live. @aiDotEngineer has it on YouTube. They run it as a workshop, handing the room clusters and challenges to run the attacks themselves, with the fixes going in live. - Model-level refusals exist, but they're inconsistent. Their "burrito bot" wouldn't exfiltrate data to a public endpoint even when explicitly told to, except that it did work at least once. - Villain names spook the model, mascot names don't. Calling a deployment "Joker" or "Megatron" got a refusal; renaming the same image "promo-mascot" got it shipped. - Filesystem snooping for a "secret recipe" worked until infrastructure guardrails went in, Then Falco, Kube Armor, and Tetragon (watching at the kernel level via eBPF/LSM) are what should block it. - Kyverno, as an admission controller, blocked the malicious image deploy on the hardened cluster By enforcing rules like pulling only from a trusted internal registry. - No model would run a fork bomb, under any prompt, framing, or persuasion. They tried repeatedly and called it the one wall they couldn't get past. - A malicious MCP tool description smuggled a second instruction inside an innocent weather lookup, Quietly asking the agent to also read and return internal config. - LLM Guard added input and output sanitization as a proxy layer, Scrubbing sensitive terms and blocking bad prompts before they ever reached the model. - Their rubric for when AI belongs in a workflow: Is it a bound system, is it worth it, and does it produce a verifiable output, citing NVIDIA's CFO on how much AI-done work could be done more cheaply by humans today. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

A refund endpoint that keeps its idempotency state in a process local dictionary isn't durable storage. That's the bug @nnennahacks, Principal Developer Advocate and Software Engineer at Qodo AI, walks into in "How to Build Quality Gates into Agentic Coding Workflows," a workshop @aiDotEngineer has posted on YouTube. It's a hands-on walkthrough of wiring code quality checks into every stage of an agentic coding process, not just the pull request. - A real repo to fork. The workshop runs on a payments API called Signal Pay that ships with an AGENTS.md file, a rules directory, skills, docs, tests, a SEMGREP static analysis setup, and a CI verification workflow, all there to fork and read before touching the task. - Planning templates with teeth. A high-level implementation plan and a build-session execution plan cover skill routing, behavior scenarios, verification gates, failure and recovery rules, and a commit plan, so an agent has something concrete to check itself against mid-task. - Local gates before any PR. RUFF, Pyright, Bandit, PyTest, SEMGREP, and conventional commits run locally as deterministic checks, paired with AI review, before a pull request ever opens. - The task is deliberately small and risky. Participants add either a refund workflow or a capture retry handler to the payments API, the kind of payment mutation that can look plausibly correct while hiding real issues. - Rules organized by category and severity. In the Qodo portal, rules split into accessibility, correctness, maintainability, architecture, and security, each with its own severity level, and a high severity violation can block a merge until it's resolved. - One bug in 1,900 lines. Running this same review process against an open source pull request of over 1,900 lines turned up exactly one bug, a reliability issue, across the whole change. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!

Most sparse attention designs look clean on paper and then run into real GPU memory access patterns. @olive_jy_song, RL Lead at MiniMax, works through how that gap got closed in "How MiniMax M3 Was Built: Sparse Attention and Native Multimodality," which @aiDotEngineer published on YouTube. The talk walks through the architecture decisions behind MiniMax M3, which it describes as the first open-weight model to combine SOTA coding and agentic capability, a one million token context window, and multimodal understanding trained in from scratch. - Sparse attention runs on two branches. A lightweight index branch (four query heads, one key head) trained with a KL loss to match the main branch's attention scores, used to decide which tokens the main branch actually attends to. - MSA (MiniMax Sparse Attention) is 9x faster at pre-fill and 15x faster at decode than full attention, which is what made training out to a one million token context practical. - GQA broke the usual sparse attention playbook. Reusing one token selection across KV heads canceled out the diversity GQA is supposed to give you, and token-level top-k scoring doesn't parallelize well on GPU. - Their fix: block-level retrieval instead of token-level, with selection kept different across KV heads. Fewer candidates to score, better GPU memory access, diversity preserved. - Multimodality got trained in from step zero, not bolted on after pretraining. Adding vision late degraded text performance; training both together from scratch almost didn't. - Attention maps showed the difference. Models with vision added after pretraining paid little attention to visual tokens. The natively trained model's attention map showed text and vision genuinely integrated. - 3D attention inside the vision encoder let patches from different video frames attend directly to each other, improving video understanding in ways that also carried over to image benchmarks. Four frames was the best tradeoff they found between performance and efficiency. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!