AI, coding, software, and whatever’s on my mind.

📩 for collaboration
Pinned Tweet
Decision models give AI agents a way to evaluate defined outcomes without generating an answer as text. The basic interface is straightforward. An application supplies a state, a question, and the possible answers. The model assigns probabilities to those answers, which the application can then use for classification, tool selection, routing, severity assessment, or other decisions. That sounds simple, but the mathematics has consequences across the entire agent system. A binary decision produces a probability that can be compared against a threshold. A categorical decision distributes probability mass across the supplied alternatives. An ordinal decision produces a distribution over ordered levels, from which an expected score can be calculated. These are different output contracts, and each requires different handling by the application. There are also important limitations. Softmax normalization does not guarantee that probabilities are calibrated. A high probability does not establish that a prediction is correct. Changing the available options changes the decision problem, and omitting the correct answer can force probability mass onto incorrect alternatives. Even when the prediction is reasonable, the appropriate action depends on error costs, thresholds, permissions, and escalation policies. I put together a technical handbook on Decision Models for AI Agents, using Drex 2.0 and Drex Max to work through the mathematics and engineering in detail. Drex 2.0 provides a useful implementation reference through its released model weights and runtime. The handbook examines how candidate answers are represented, how hidden-state readouts are used in pointer-head scoring, how logits become probabilities, and how question execution can reuse shared state. Drex Max extends the typed decision interface to images. That introduces another set of problems around image preprocessing, context limits, visual ambiguity, and incomplete evidence. The handbook keeps its documented behavior separate from internal architectural details that have not been publicly established. The material works through everything from first principles => binary, categorical, and ordinal decisions, probability distributions, expected risk, cost-sensitive thresholds, abstention, pointer-head mathematics, temperature scaling, shared-state inference, multimodal inputs, calibration, evaluation, and production failure modes. The explanations include worked mathematical examples, with Drex-specific implementation details grounded in Nace AI's official code and documentation. Sharing the handbook here 👇
13
4
38
1,758
Decision models give AI agents a way to evaluate defined outcomes without generating an answer as text. The basic interface is straightforward. An application supplies a state, a question, and the possible answers. The model assigns probabilities to those answers, which the application can then use for classification, tool selection, routing, severity assessment, or other decisions. That sounds simple, but the mathematics has consequences across the entire agent system. A binary decision produces a probability that can be compared against a threshold. A categorical decision distributes probability mass across the supplied alternatives. An ordinal decision produces a distribution over ordered levels, from which an expected score can be calculated. These are different output contracts, and each requires different handling by the application. There are also important limitations. Softmax normalization does not guarantee that probabilities are calibrated. A high probability does not establish that a prediction is correct. Changing the available options changes the decision problem, and omitting the correct answer can force probability mass onto incorrect alternatives. Even when the prediction is reasonable, the appropriate action depends on error costs, thresholds, permissions, and escalation policies. I put together a technical handbook on Decision Models for AI Agents, using Drex 2.0 and Drex Max to work through the mathematics and engineering in detail. Drex 2.0 provides a useful implementation reference through its released model weights and runtime. The handbook examines how candidate answers are represented, how hidden-state readouts are used in pointer-head scoring, how logits become probabilities, and how question execution can reuse shared state. Drex Max extends the typed decision interface to images. That introduces another set of problems around image preprocessing, context limits, visual ambiguity, and incomplete evidence. The handbook keeps its documented behavior separate from internal architectural details that have not been publicly established. The material works through everything from first principles => binary, categorical, and ordinal decisions, probability distributions, expected risk, cost-sensitive thresholds, abstention, pointer-head mathematics, temperature scaling, shared-state inference, multimodal inputs, calibration, evaluation, and production failure modes. The explanations include worked mathematical examples, with Drex-specific implementation details grounded in Nace AI's official code and documentation. Sharing the handbook here 👇
13
4
38
1,758
You can literally play with fluid dynamics in your browser. Pavel Dobryakov built a real-time fluid simulation that runs on the GPU with WebGL. Click and drag and you can push dye through the velocity field, create vortices, mix colors and watch the flow evolve in real time. The implementation is based on GPU fluid-simulation techniques for the Navier-Stokes equations. No install. Works on mobile too. paveldogreat.github.io/WebGL…
2
5
37
2,245
You are lucky if you work with a senior engineer who never makes you regret asking a basic question. You ask something you think everyone else already knows. They explain it without showing off. Then somehow you leave understanding more than what you originally asked. Those people are rare. And you remember them for years.
3
1
25
1,451
LLM quantization reduces the precision used to represent model weights, activations and KV-cache tensors. The goal is to reduce memory requirements and data movement while keeping numerical error within acceptable limits. But bit width alone doesn't determine accuracy, memory consumption or inference speed. There are several important engineering decisions. 1./ Quantization formats INT4, INT8, NF4, FP4 and FP8 use different numerical representations. Uniform integer quantization maps values through scales and, optionally, zero-points. Floating-point formats have exponent and fraction fields, while NF4 uses a nonuniform codebook. The same bit width can produce different quantization errors. 2./ Scaling and granularity Quantization can operate per tensor, per channel or per group. Smaller groups can reduce reconstruction error, but require additional scaling metadata and compatible kernels. Rounding and clipping introduce error. Outliers make this particularly important. 3./ Quantization algorithms GPTQ: Uses approximate second-order information to reduce errors introduced by weight quantization. AWQ: Uses activation statistics to identify salient weight channels and protect them during quantization. SmoothQuant: Redistributes quantization difficulty between activations and weights to enable W8A8 quantization. These methods address different aspects of the quantization problem. 4./ Training approaches PTQ: Quantizes an already-trained model, often using calibration data. QAT: Accounts for quantization effects during training, commonly using fake-quantization operations. QLoRA: Keeps a quantized pretrained base model frozen while training low-rank adapters. 5./ Memory accounting For a model with exactly 8 billion weights: FP16 weights = 16 GB Packed INT4 weights = 4 GB One FP16 scale per 128 weights = 0.125 GB Total = 4.125 GB for weights and scales, assuming no additional quantization metadata. KV cache, activations and temporary buffers are separate memory costs. 6./ Storage versus computation A W4A16 model uses 4-bit weights and 16-bit activations. That doesnt necessarily mean the matrix multiplication itself runs in 4-bit arithmetic. Some kernels unpack and dequantize weight tiles before performing FP16/BF16 operations. Accumulation precision is another choice. The actual speedup depends on kernel support, dequantization overhead, memory bandwidth, batch size and workload characteristics. 7./ Evaluation Quantization needs to be evaluated for both model quality and serving performance. Measure held-out task accuracy, numerical stability, time to first token, time per output token, throughput and peak memory under comparable conditions. A 4-bit model isn't automatically four times faster than its 16-bit counterpart. Quantization determines how values are represented. The hardware, kernels and workload determine how efficiently those values are processed.
10
23
198
4,506
Attention explained!!
I put together a handbook on how attention works, from the Q/K/V equation up through how it actually runs in modern LLMs. Self vs. cross-attention, causal masking, multi-head attention, RoPE, KV caching, MHA/MQA/GQA/MLA, FlashAttention, worked through by hand, with citations back to the original papers. Made it to be the explanation I wish I'd had.
6
58
424
14,398
4 free CUDA resources to learn GPU programming from scratch. 1./ NVIDIA - Modern CUDA C++ Learn GPU programming directly from NVIDIA. Video lectures, slides, and coding notebooks covering parallel algorithms, CUDA streams, and writing your own kernels. github.com/NVIDIA/accelerate… 2./ freeCodeCamp - CUDA Programming 12 hours with Elliot Arledge. Starts with setup and a C/C++ review, then moves into kernels, memory management, matrix multiplication, and custom PyTorch operations. freecodecamp.org/news/learn-… 3./ University of Illinois - Applied Parallel Programming Understand CUDA’s execution and memory models, then work through tiling, reductions, scans, and convolution. lumetta.web.engr.illinois.ed… 4./ University of Oxford - CUDA Programming on NVIDIA GPUs Free lecture slides and practical exercises from Mike Giles and Wes Armour. Covers memory, synchronization, libraries, and programming across multiple GPUs. people.maths.ox.ac.uk/~giles…
7
55
221
6,636
Tech with Mak retweeted
Learn Harness Engineering 🚀 Found this excellent open-source course by Walking Labs, covering 14 lectures and 8 hands-on projects on building more reliable AI coding agents. What's inside: → Agent instructions (AGENTS.md / CLAUDE.md) → Context management and persistent memory → Multi-session continuity and state tracking → Task boundaries and scope control → Testing, verification, and observability → Automated agent loops and workflows → Multi-agent orchestration and graph engineering The practical projects walk you through building a complete agent harness, comparing prompt-only execution with structured workflows. Also includes ready-to-use templates, verification scripts, and a harness-creator skill. Perfect for developers working with Claude Code, Codex, and other AI coding agents who want to understand the engineering behind reliable agent execution. Completely free. Here's the GitHub Repo: github.com/walkinglabs/learn…
39
221
1,143
44,227
Somebody just built the Truman Show for a fly.
We placed the fruit fly brain inside Odyssey-3, a frontier world model. The lil’ guy is now flying around totally imagined, simulated worlds all by itself 🫶
1
10
2,133
A geophysicist got laid off, built his own AI job-search system using Claude Code, and landed a job as an AI engineer. 69 tailored applications. 20 first-round interviews. One signed contract. The developer, Mads Lorentzen, has now open-sourced the workflow, and the GitHub repository has already crossed 50,000 stars. Its called AI Job Search, and it handles much more than rewriting resumes. The system uses Claude Code to manage different stages of the job search: → /setup builds a structured profile from your CV, work history, skills, and career preferences. → /scrape searches supported job boards, removes duplicates, and ranks openings by fit. → /apply evaluates a job description and generates a tailored CV and cover letter. → /interview prepares you using the actual job posting and documents you submitted. The application workflow is particularly well designed. A second Claude agent reviews the CV and cover letter in a fresh context, checks them against the job requirements, and suggests revisions. The documents are generated using LaTeX, compiled into PDFs, and checked for layout problems and readable text extraction to improve compatibility with applicant tracking systems. Theres also application tracking, skill-gap analysis, and support for adding custom job portals. The framework doesnt blindly submit applications. You choose which opportunities to pursue, and the current job-board integrations are strongest in Denmark, with options for other markets. One important detail => the repository recommends keeping your personalized setup private, since it contains your employment history and contact information. Fully open source. Here's the GitHub Repo: github.com/MadsLorentzen/ai-…
21
14
138
6,646
I thought I understood GPU utilization until I started working on this handbook. I kept collapsing three different things into one mental picture: > the work we launch, > the work that is actually resident on the GPU, and > the work that is ready to issue right now. They are not the same thing. Take a CUDA kernel with 240 blocks and 256 threads per block. That is 61,440 logical threads, or 1,920 warps. But that launch count does not tell you how many warps are resident at once. Blocks are admitted onto SMs as registers, shared memory, thread limits, warp limits and block limits allow. Even after a warp becomes resident, it may still be waiting on memory, an arithmetic dependency or synchronization. The scheduler can only choose from warps that are actually eligible to issue. That changed how I read performance numbers too. Occupancy is a residency number: resident warps per SM compared with the maximum supported per SM. GPU utilization measures something different. In NVML, the GPU utilization metric is the fraction of the sampling window during which at least one kernel was executing. So a GPU can show 100% utilization without Tensor Cores being saturated, without HBM bandwidth being saturated, and while one subsystem is the bottleneck and large parts of the GPU remain underused. Memory has the same kind of traps. Registers, shared memory, L1, L2 and HBM are not just increasingly slower boxes. They have different scope, capacity, management and access behavior. A model fitting in HBM tells you that it fits. It tells you almost nothing about how many bytes move, whether accesses coalesce well, how much reuse you get, or whether memory is what is holding the kernel back. FlashAttention is a nice example. It computes exact attention, but uses tiling and recomputation to reduce traffic between HBM and on-chip storage. Tensor Cores were another thing I had mentally oversimplified. They are specialized matrix multiply-accumulate hardware and they matter enormously for AI, but they do not “run the model.” Reductions, indexing, elementwise work, synchronization and memory movement still involve other parts of the GPU. Kernel launches add overhead too. After a while I stopped asking "why isn’t the GPU at 100%?" and started asking a better question: what is actually limiting useful progress right now? That ended up becoming a 42-page handbook. It covers SMs, warps and schedulers, memory and latency hiding, Tensor Cores, GEMM mapping, precision, and the performance numbers that are very easy to misread. Full handbook in comments.
7
31
212
5,608
You can read it here: drive.google.com/file/d/17ty… For more such handbooks subscribe to my Free Newsletter. thecuriousmak.substack.com/s…
1
4
492
Richard Feynman called Euler's formula "the most remarkable formula in mathematics." And there's a fascinating engineering reason why this equation is so important. e^(iθ) = cos(θ) + i·sin(θ) It connects exponential functions, imaginary numbers, trigonometry, and rotation through a single mathematical relationship. But its usefulness goes much further. Consider a sinusoidal signal: x(t) = A cos(ωt + φ) Using Euler's formula, we can represent it as the real part of a complex exponential: A e^(i(ωt + φ)) Why would engineers use this representation? Because differentiating e^(iωt) simply multiplies it by iω. Phase shifts can be represented through complex multiplication. For linear, time-invariant electrical circuits in sinusoidal steady state, this allows differential equations to be reduced to algebraic equations. Engineers use this mathematics in phasor analysis, AC circuit calculations, impedance, and frequency-response analysis. The same complex-exponential representation is fundamental to Fourier analysis, signal processing, communication systems, and the study of mechanical vibrations. Feynman also described the complex-plane representation of numbers as the unification of algebra and geometry. And that's what makes Euler's formula so remarkable. An equation connecting imaginary numbers and circular motion provides a practical way to analyze real electrical signals, oscillations, and physical systems.
30
183
906
41,221
Someone built an open-source CapCut alternative that AI agents can actually control. 🤯 Its called Concat, and the engineering behind it is pretty impressive. A full video editor built around a native Rust engine, with GPU-accelerated rendering and an API for automated editing. Here's what it supports: → Multi-track editing with keyframes and transitions → Automatic captions using local Whisper models → Text-to-speech and voice cloning → AI background removal → Voice cleanup and audio effects → 4K video export at up to 60 FPS And here's the interesting part. Concat exposes a JSON-RPC API, so scripts and AI agents can create projects, import footage, arrange clips, add titles, and export videos programmatically. The AI models run locally after downloading. Your footage doesnt need to be uploaded to a cloud service. Use it for free, with no subscriptions, watermarks, or account required. Available for macOS, Windows, Linux, and Android. Still in beta, but completely open source under AGPL-3.0. Here's the GitHub Repo: github.com/jub0t/Concat
13
16
91
4,002
10 free resources for learning LLM inference engineering. How do you make an LLM serve more users, respond faster, and fit within GPU memory? These resources cover the concepts, code and measurement behind it:
5
64
425
12,917
9./ Triton - Official GPU Programming Tutorials Go from vector addition to softmax, matrix multiplication and fused attention. Runnable examples for learning how GPU kernels work. triton-lang.org/main/getting…
1
1
366
10./ NVIDIA AIPerf - Benchmarking Guides Measure first-token latency, token latency, throughput and goodput. Includes guides for controlled loads and realistic workload traces. github.com/ai-dynamo/aiperf Basic Python, PyTorch and Transformer knowledge will help. The learning materials are free to access. Running the examples may require paid compute.
3
359