DevSecOps engineer · Kubernetes (CKA/CKAD/KCSA) Local AI on a DGX Spark: benchmarks, recipes, breakages Open to work & collabs · 🇮🇹 in 🇨🇭

Seven AI models. One DGX Spark. One local workspace. 🔥 Sparkfire puts a full AI suite on your own NVIDIA DGX Spark: read documents, transcribe recordings, translate, generate images, make videos with sound, sing songs from your lyrics, build mind maps, and chat about all of it with web search. Everything runs locally, side by side, behind one interface. 𝗪𝗵𝗮𝘁 𝗿𝘂𝗻𝘀 𝗼𝗻 𝗼𝗻𝗲 𝗦𝗽𝗮𝗿𝗸 👇 💬 Chat, mind maps, web search · Saluki 27B · 12.7 GiB 📄 Read documents (OCR) · PaddleOCR-VL-1.6 · 5.4 GiB 🎙 Transcription + subtitles · Parakeet-TDT-0.6B-v3 · 1.5 GiB 🌍 Translation, 38 languages · Hy-MT2-1.8B-FP8 · 2.1 GiB 🖼 Image generation · Qwen-Image-2.1-Turbo · 17.5 GiB 🎬 Video with sound · FastH3 · loads on demand 🎵 Songs from your lyrics · YuE2-3B · loads on demand 🔎 Web search · SearXNG, running locally Everything loaded and idle: 51 of 121 GiB. Video and music load when you use them. 𝗪𝗵𝘆 𝗼𝗻𝗲 𝘄𝗼𝗿𝗸𝘀𝗽𝗮𝗰𝗲 No juggling seven apps, seven interfaces and seven API bills. Every result is a plain file with its history, so one feeds the next: a scanned PDF becomes Markdown, then a translation, a mind map, a summary or a conversation. A generated image becomes the first frame of a video. 𝗪𝗵𝗮𝘁 𝗶𝘁 𝗱𝗼𝗲𝘀 📄 OCR that keeps headings, tables and reading order: 99.9% of words in order on a 20-page test, about 6 s a page 🎙 Transcripts with one line per sentence, SRT and WebVTT subtitles; click a line and the player jumps there 🌍 Translation as you type, or whole documents at once 🖼 Images in about 18 s at 1024×1024, about a minute at 2K, with legible lettering 🎬 5, 8 or 10 s videos with their own soundtrack; 5 s at 480p in about a minute 🎵 48 kHz stereo songs from your lyrics, with an editable score 💬 Chat with your PDFs, images and code, live reasoning, and answers that keep going when you close the tab 🔎 Web search with numbered sources 🔀 Workflows in a node editor, prompt presets, and a gallery with side-by-side compare 📱 Dark and light themes, and it works on your phone 𝗟𝗼𝗰𝗮𝗹 𝗯𝘆 𝗱𝗲𝘀𝗶𝗴𝗻 The models run on your Spark. Your files live in ordinary folders you can read, back up or move. No sign-up, no API keys, no per-token bill. 𝗚𝗲𝘁 𝗶𝘁 𝗿𝘂𝗻𝗻𝗶𝗻𝗴 git clone github.com/MovieMaker93/Spar… ./scripts/start-dgx-spark.sh --accept-model-licenses A fresh install took 20 minutes and 106 GB of disk. Open source (Apache 2.0). Sparkfire is a fork of @stevibe 's SparklingKit (github.com/stevibe/Sparkling…), now grown into its own project. Thankou, Steven 🙏 Model licenses apply: FastH3 isn't licensed in the EU, UK, South Korea or US (install with --video-backend none); Qwen-Image and YuE2 are non-commercial. ⭐ github.com/MovieMaker93/Spar…
5
3
13
54,020
I hate speed claims so much. Quality should always be a top priority. I also think most recipes deliberately hide the real speed numbers because most people only care about speed. So, claiming “C1 = 80 tokens/s” without specifying whether that’s for code or prose, or how heavily you quantized the KV cache, is criminal behaviour. 😂
The more I dig in the various inference engines out there, the more I see the shortcuts they take for speed. Don't assume that model layers are mapped 1-1 into your engine, some are quantizing before running, 8bit and even 4bit. I think I should post more on this, not to finger point anyone, but just to clarify that: more often than not, you bargain precision for speed. Kernel optimizations are great, but there is a physical limit (compute/memory bandwidth). There is no free lunch 🤷🏻‍♂️
1
2
161
Really solid test. While fine-tuning a small model, I quickly realized how deeply the harness affects the model, from tool calling to overall quality. I think we should build a good benchmark that evaluates a model's overall performance across every possible harness. PS: I'm using opencode with DeepSeek and it's working really well (opencode offers DeepSeek 4.1 Flash for free)
I repeated DeepSeek V4.1 Flash 1017 benchmark 89 times across 9 harnesses to understand which works best. High effort, 5 attempts each: Pi harness: 40.4 Grok Build CLI: 40.4 DeekSeek: 39.2 OpenCode 2.0: 38.6 OpenCode: 36.6 Claude Code: 36.4 Codex: 33.2 Hermes Agent: 32 Kilo Code: 31.8 Codex kept failing several times exceeding the allowed time per repo (3 hours) and was judged where it finished. Based on what I tested, Pi, Grok Build CLI, and OpenCode 2.0 look like the strongest universal harnesses. This doesn't necessarily generalize. A detailed table with every run in the response🧵
1
103
Love this acquisition For me, the best use of AI is to reduce suffering and search for new cures. Looking forward to this!
We’ve acquired Orchestra Bio, a platform that helps biopharma teams coordinate the complex process of drug development. Together, we’ll build on Rosalind, our AI models and tools for life sciences research, to streamline those operations and help new treatments reach patients sooner. Welcome @ashokaraj_ and team! openai.com/index/openai-to-a…
47
Qwen3.8 Flash Next FP4 on one DGX Spark, same image: • Prefill +4.5%: a 128k prompt now runs at 2,114 t/s, first token in 61.6 s (was 65.4), output bit-identical • KV cache 1.37M → 1.58M tokens: 6 full 262k requests at once, up to 8 in parallel This is the best quality recipe on a single Spark. Next: working on adding an MTP vocabulary drafter to improve decode for specific languages, and dynamic MTP. github.com/devopsfortunato/Q…
1
2
136
Even though it applies only to federal agencies, it’s a signal that the “AI war” has begun. It’s only a matter of time before the US blocks Chinese AI models, and perhaps even restricts access to its own closed-source models. Today, it’s more urgent than ever for Europe to wake up and build its own AI. We cannot rely on foreign countries to provide us with such a valuable strategic asset. #EuropeWakeUp
Was only a matter of time. DeepSeek now, GLM and others later.
72
Italians doing it better 🇮🇹💪🏻
If you want a Local AI super star to follow here he is @mudler_it 🚀 He is the creator of LocalAI project that has ONLY 50K stars on github. Keep pushing Ettore! Link: localai.io
2
3
194
This is not a good example of italian talent :D
35
Same. Free API for 3 days is nice, but the weights are what matter. If they land on the 23rd at Flash size, it goes straight onto the Spark for a test.
12
NVIDIA released a new quantization for Qwen3.8-Flash-Next (info: developer.nvidia.com/blog/ho…), so I spent two days looking for the best balance between quality and speed on my DGX Spark. My rule: quality comes first. On a small model, output quality matters far more than raw speed numbers. I tested my recipe against TensorFold by @MiaAI_lab and hibrid48 by @vr8vr8 / @myllmbox, the quality-focused option, using: - tool-eval-bench by @SeraAndroid - sixcat-eval by @ViC305 - llm_context_benchmarks by @ivanfioravanti, up to 128K tokens The short version: answer quality is close. Speed, memory and structured output are not. Full breakdown: nitter.siddz.com/devopsfortunato/status… I also released my recipe with two presets. Fast is the default. Quality improves output fidelity at the cost of some prefill speed. github.com/devopsfortunato/Q…
2
1
6
322
you might be interested to see the results of the differente recipes @AgentSparko
1
1
79
Pre-bed reading. NVIDIA does it again.
Replying to @jnardiello
The FP4 is using quantization-aware distillation (QAD) Read here: developer.nvidia.com/blog/ho…
81
Spent the day squeezing Qwen3.8-Flash-Next FP4 on one DGX Spark. For me this is the best-quality quant of Qwen3.8-Flash-Next you can run on a single DGX Spark. 1M context, every needle found. Up to 1.7M tokens of KV cache. Tool calling holds at 91–92/100. 2M context test runs tonight. Recipe update soon 👀 github.com/devopsfortunato/Q…
1
1
115
Wow, solid release from DeepSeek I've always loved the way DeepSeek models answer. Their responses always feel so satisfying and well-grounded. Eager to try this new beast
NEW DeepSeek v4.1 Flash-1017 is LIVE!! 🔥 Looks like it improved on everything! Link below.
67
I tried the new mysterious model on opencode (LL Alpha Free) with the classic pagoda prompt. Really disappointing outcome, even on max thinking 😕 Prompt: "Design and create a very creative, elaborate, and detailed voxel art scene of a pagoda in a beautiful garden with trees, including some cherry blossoms. Make the scene impressive and varied and use colorful voxels. Use whatever libraries to get this done but make sure I can paste it all into a single HTML file and open it in Chrome." It didn't even run: the clouds had no colour set, so the page crashed on load and showed a blank sky. After a one-line fix it renders, but the pagoda roofs float as separate layers with little structure between them.
2
246
Been testing Slate AI Studio on my DGX Spark and the interface is really good. Tried the image feature with Qwen-Image-2.1 Turbo: insane quality for the speed, ~33 seconds per portrait. Three examples below 👇 Still testing the rest, but the image side already impressed me. Built by @u1tra_instinct, powered by TensorFold from @ashxhart github.com/drowzeys/keys-Was…
3
124
Small Grok Bot, but useful for me: Switchboard 🔀 If you run a @LiteLLM proxy with different local models, you know the pain of setting everything up again every time you swap. Switchboard fixes that. It keeps the same alias for the same model family, so when you swap recipes (vLLM, llama.cpp, Ollama…) you configure your harnesses once and never touch them again. What it does: • finds the model that's running and reads its id • points the family alias at it, or adds a new one • backs up the config before every change • reloads LiteLLM and tests a real chat completion • rolls back automatically if anything breaks It only edits the LiteLLM config. It never starts or stops your model servers. Built for my DGX Spark. Still early and I'm still testing it. Try it: x.ai/bot/ppO9XYR2CUS7-Hyaklf… #LocalAI #LiteLLM
2
1
4
236
Qwen3.8-Flash-Next FP4 on a single DGX Spark, one command. Early recipe, still testing. ./start.sh serves NVIDIA's NVFP4 quant over an OpenAI-compatible API. Settings: • vLLM in eugr's Spark image, b12x kernels for GB10 • 262k context, FP8 KV cache • Up to 8 parallel sessions • MTP speculative decoding, 4 draft tokens • PLE tables offloaded to disk • Qwen3 reasoning + tool-call parsers What's NVFP4? NVIDIA's 4-bit float format: every block of 16 weights shares an FP8 scale, plus one scale per tensor, so it holds up better than plain INT4. ModelOpt "mixed" keeps sensitive layers at higher precision. ~96 GB, native on Blackwell FP4 tensor cores. Numbers (single stream), vs MiaAI-Lab's TensorFold INT4 recipe on the same GB10: • 256k ctx: 1,881 vs 1,970 t/s prefill, 35 vs 38 t/s decode • tool-eval-bench: 90/100 vs 85 • Structured output: 83% vs 40% • Hard mode: 91% vs 74% • Tool-call TTFT: 245 ms vs 526 ms • TensorFold decodes faster up to 32k; FP4 holds ~35 t/s at every depth Recipe: github.com/devopsfortunato/Q… TensorFold recipe by @MiaAI_lab Thanks @NVIDIAAI, vLLM, eugr
5
5
268
@ivanfioravanti have you tested the new quant from nvidia?
62