AI Engineer • running models locally until something catches fire llama.cpp • vLLM • GGUF • homelab
- Tweets 1,178
- Following 68
- Followers 862
- Likes 657
I built my own network topology designer using DeepSeek.
It's called NetForge.
I wanted something like Draw.io, but focused on actual networking equipment, realistic hardware models, and infrastructure visualization.
Here's what I've been building. 🧵
1
41
I'm working toward a more complete infrastructure designer with better hardware models, cabling, rack layouts, and eventually live network discovery.
Built with DeepSeek, one phase at a time.
What features would you add?
1
6
Might be late for this, but here is a Pagoda generated by DSV4.1 Flash running on Project Maya!
It is the DSV4.1 Flash EXL3 3.5 Bit quant by libertAIDAI
1
3
278
Need More VRAM retweeted
OrcaRouter x Project Maya
Project Maya × @OrcaRouter 🐋
OrcaRouter's OrcaCyber DeepSeek-V4.1-Flash-1017, the abliterated model built for cybersecurity research, now runs on Maya.
Both versions, Mix-256G and Mix-128G, are in Maya's model picker. One command sets it up on your own GPUs: NVIDIA or AMD, Linux or Windows.
Prefer an API? Use it on OrcaRouter: orcarouter.ai/models?tier=cy…
Thanks to the OrcaRouter team for working with us on this.
3
3
53
4,746
With my X account I can earn up to $95 a month. How much can you earn? creators.unfair.so/worth/nee…
129
Need More VRAM retweeted
Project Maya v1.1.29 is here, and it's a big one: MiniMax-M3.1-Flash now runs on Maya.
That makes three frontier open models you can run at home on your own GPUs: GLM-5.3-Flash, DeepSeek-V4.1-Flash and now MiniMax-M3.1-Flash, a 428B model with 23B active per token and up to a 1M context.
MiniMax gets its own engine, built by @needmorevram. We brought it up to the same standard as GLM and DeepSeek before shipping it:
- Runs on NVIDIA from V100 up, on Linux and Windows
- Fits itself to your PC: the setup picks the context your memory can hold, and the engine plans VRAM and RAM instead of failing
- A compact cache by default, so long contexts take half the memory with no loss in quality
- Pictures, tool calls and every one of MiniMax's thinking levels, with its own recommended settings out of the box
- The vision tower stays off your GPU until you send a picture, so the VRAM goes to the model
Huge thanks to @needmorevram, and to everyone who tests, reports and contributes to Maya. This is what community open source looks like.
Next up (WiP): our own Maya GGUF quants of MiniMax-M3.1-Flash, small enough for a 128 GB PC.
Link in the reply 👇
18
34
174
10,842
Need More VRAM retweeted
I thought a lot about whether to put energy into this. At this point I have no option other than to defend my work.
After this, I'm off to enjoy the last weekend of my paternity leave before going back to work.
12
4
71
5,968
Will be interesting with maya 👀
Looks like EXLR8 on GLM 5.3 flash is going to be a winner too. Uncalibrated, errors are ~17% lower. With the calibration pass, that'll drop even more.
More news later this week, but expect the same large weight savings that we got from EXLR8 in GLM 5.3 full without any compromise on model quality.
And don't worry: EXLR8 is engine-agnostic. Coming soon to your fav engine. (and of course to github.com/kindlingai/gr8)
1
2
629
25-30 Tok/s and 600 Tok/s prefill on coding tasks with a single 3090 running EXL3-4.05 with project Maya 👀
10
6
38
3,131
Time to give this a try! 👀
Introducing Project Maya 🟣
GLM-5.3-Flash - a 321B parameter model - running on your own GPU.
⚡ Up to 40 tokens/s answering, up to 440 tokens/s reading your prompt (2x Tesla V100 + 30 GB RAM)
🧠 The whole 321B model in a 90 GB quant made from Z.ai's own FP8 weights
📏 Steady speed as the conversation grows: Maya keeps its pace with context depth - a 60K-token conversation answers about as fast as a short one.
How it works: Maya's engine keeps the most-used experts on your GPU, the next ones in RAM and the rest on your SSD, and moves them as the conversation needs them. It adapts to whatever hardware it finds - one GPU or two, more or less RAM. It runs with 32 GB of RAM; more memory is faster.
What you get:
• 🖼️ Pictures - the vision encoder only loads while it reads a new image, so the model keeps the whole GPU the rest of the time
• 🔌 OpenAI- and Anthropic-compatible API - plug in your apps and coding agents
• 🚀 MTP speculative decoding with the model's own draft layer
• 🧩 Our own quants (Maya-S), measured against the full FP8 model
• 💬 A chat dashboard with live monitoring of the model and your hardware
• 🛠️ One-command install: git clone, ./setup.sh, answer a few questions - it compiles the engine for your GPU, downloads the model (checksum-verified) and starts
Every machine is different - try it on yours and tell us what you get.
Built on Strata (MIT) and ggml/llama.cpp. 🙏
5
13
1,281