Structurally cheaper inference for open models.
- Tweets 10
- Following 3
- Followers 138
- Likes 39
Let there be tokens 馃殌
The Liquid Inference router network is growing quickly. We now have 28 verified companies serving >1000 models, offering capacity dynamically and updating quotes in real-time. Average time for providers to onboard and integrate models is 3 hours, full verification within a day.
4
11
381
Most inference GPUs are ~70% idle. It's not a kernel problem, or a traffic problem. It's a placement problem.
To prove it, we built an inference service that co-locates models on a GPU.
5 co-located models. 14 days. 2.5B tokens served. $17k/mo saved.
4
1
13
1,367
Today, we are launching our collaboration with @nomic_ai to make AI agents more effectively and efficiently understand complex PDF documents.
Nomic's new nomic-layout-v1 model allows your AI agents to parse documents locally, so sensitive documents never leave your machine.
2
6
17
3,131
