Distributed inference for the people. Private and public mesh, sharing hardware for inference. github.com/mesh-llm/mesh-llm
github.com/mesh-llm/mesh-llm
- Tweets 9
- Following 60
- Followers 37
- Likes 0
New Mesh plugins for Apple Silicon:
⚡ Uzu: high-throughput Metal inference for Qwen 3.8 & more
🧠 DwarfStar: run huge DeepSeek V4 models locally
💾 Slotstream: SSD-backed MoE for Macs with less RAM
github.com/Mesh-LLM/uzu-runt…
github.com/Mesh-LLM/ds4-plug…
github.com/Mesh-LLM/slotstre…
1
1
1
16
this was fun to build
- fork of signal
- agent built with GDK
- using open weights model
- served via e2ee mesh-llm
- via private GPU TEE
- making private search tool calls
- paid for with ₿ over LN
a cool stack of permissionless & open source tech all working together!
An agent inside Signal using decentralized compute paid for with bitcoin
@moneyball demos a fork of Signal built by @matbalez using @Mesh_LLM and @lexeapp ↓
Timestamps:
0:00 - Demo: agent inside Signal
10:51 - The economics of decentralized compute
14:39 - Running inference at home
13
24
155
161,234
Right on - this is what we made MeshLLM for (well that and some procrastination: github.com/mesh-llm/mesh-llm
Metal, CPU, CUDA (various), ROCm, Vulkan, x86_64, arm, windows/mac/linux and appropriate combinations.
(fingers are sore).
this is THE time to be revisiting distributed inference on heterogeneous hardware, software is collapsing the barriers of hardware incompatibility
2
2
48
we have shifted a fair bit of dev to buzz to try out multi player agents, just to see how it goes. Some findings from the other month: engineering.block.xyz/blog/s…
1
37