Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → sglang.io/summit 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and the systems behind them. Featuring @LipBuTan1 (Intel), @lilianweng (OpenAI), @soumithchintala (Thinking Machines), @AravSrinivas (Perplexity) and researchers and builders from across the AI ecosystem. Plus hands-on workshops on model training and serving decision models, hardware Expo, and plenty of community events. What does AI infra mean to you?

Large-scale RL has a problem: weight updates from trainers to samplers are slow and expensive We’re sharing weight teleportation: faster weight updates than existing intra-cluster techniques with internet speeds, enabling samplers anywhere (maybe even space!) Weight teleportation is so efficient (<0.23s sampler pause time, ~3s non-blocking compression + prepare), the bottleneck ends up being the GPU hardware’s decompression ASIC We unlock these gains by using GPUs for massively parallel compression of model weight changes, engineering a heterogeneous decompression and patching pipeline, along with other optimizations and overlaps. These techniques are co-designed with hardware-native features in NVIDIA’s Blackwell generation of chips We’ve open-sourced weight teleportation, so everyone can use it today

Lean Verified Tokenizers (srush.github.io/verified-tok…) Training-inference mismatch kills RL runs, but yet we build messages in Jinja? Prototypes a Lean-verified Rust agent pipeline.

We accelerated one of the most optimized NVFP4 decode kernels while also making it more accurate Fewer bits typically mean more speed but less accuracy. We did the opposite: more bits, but up to 10% faster generation Already used in Ampersand’s RL and Boba serving

Space Bunny is here! 🐰 MiniMax M3.1 Flash hits >1K tok/s, bs1 decode with SGLang, production-ready from Day 0. This integration is a joint effort by @MiniMax_AI, @NVIDIAAI, @AMD, and @radixark, and you can also run RL with Miles! -Broad hardware support: NVIDIA Blackwell and Hopper (GB300/B300/B200/H200), plus AMD Instinct MI350X and MI355X. -Agentic serving at scale: ~100K total tok/s per GPU on the AgentX coding-agent trace with 96 concurrent sessions. -Speculative decoding: Supports MiniMax's DSpark draft and a DFlash2 draft trained by @radixark; supports HiCache and multimodal input. -Ultra-low-latency inference: MegaKernels deliver 2.15× faster single-request decode.

Today, in close collaboration with @awscloud and alongside the TorchNeuron launch, we’re announcing the beta of the SGLang AWS Trainium plugin, built on TorchNeuron and running on Trainium2 and Trainium3. What’s in the beta: - Inference serving and RL rollouts - Support for Qwen3, DeepSeek, and GLM model families, with more frontier models on the way - Early preview of heterogeneous RL, with rollouts on Trainium and training via Miles (@radixark) Stay tuned for more at the SGLang Summit next month!

Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now: github.com/sgl-project/sglan…

We have been working on making Beam easier to customize at @reflection_ai. Seeing the recipes and integrations go live feels pretty amazing. We're putting the code, training configs, and examples out there so people can start building on Beam. Here's what's in the cookbook: Data generation: Use Beam to create training data, including software-engineering trajectories that pass tests and reasoning examples for tool calling. github.com/reflection-oss/co… Fine-tuning: Adapt Beam to your data and tasks with LoRA, QLoRA, or full fine-tuning. github.com/reflection-oss/co… Distillation: Use Beam to teach a smaller model. We've included recipes for coding trajectories and tool-calling reasoning. Coding: github.com/reflection-oss/co… Tool calling: github.com/reflection-oss/co… Reinforcement learning: Post-train Beam against verifiable rewards. github.com/reflection-oss/co… Huge thanks to our partners and the teams behind these integrations: • NeMo AutoModel, @NVIDIAAIDev: SFT with LoRA and full fine-tuning. docs.nvidia.com/nemo/automod… • Megatron Bridge, @NVIDIAAIDev: Large-scale full fine-tuning. github.com/NVIDIA-NeMo/Megat… • Axolotl, @axolotl_ai: LoRA, QLoRA, and full fine-tuning. github.com/axolotl-ai-cloud/… • HF TRL, @huggingface: Training smaller models on Beam-generated data. github.com/huggingface/trl • Miles, @radixark: RL post-training. miles.radixark.com/docs/mode… • NeMo RL, @NVIDIAAIDev: RL post-training on NVIDIA hardware. github.com/NVIDIA-NeMo/RL/bl… • SkyRL, @NovaSkyAI: LoRA and full-parameter RL. docs.skyrl.ai/docs/examples/… Each recipe has its own instructions, dependencies, sample inputs, and smoke test. I'm excited to see people run these experiments, adapt Beam to their own work, and contribute recipes we haven't thought of. Huge thanks to @Nazeri2010, @bhutanisanyam1, @kathleen_mhatre, @FGbelidji, @joespeez, and to Thibaut, Matthias, Travis, Kai, and Yoni for making this happen. Explore the cookbook: github.com/reflection-oss/co… Access the model: platform.reflection.ai Documentation: developers.reflection.ai Provide feedback on Discord: discord.com/invite/PxUfdsNny