llamafile is the easiest and fastest way to run open LLMs on your own computer. An open source project from @MozillaAI. llamafile.ai
A latent space
- Tweets 53
- Following 18
- Followers 864
- Likes 28
llamafile retweeted
Frontier labs should be open sourcing their old models.
Businesses build on specific models with specific guardrails and specific evals, then deploy with performance expectations.
Rug-pulling that access has real impact.
1/
2
8
13
1,236
llamafile retweeted
For the last couple months I've been building a fully open source LLM performance benchmark and repository - LocalScore!
It measures how well YOUR computer performs on various LLM tasks, and creates a resource for the entire Local LLM community.
Try it: localscore.ai/download
2
7
19
2,255
Introducing LocalScore! It turns Llamafiles into "GeekBench for local AI." Benchmark open models on your local GPU or CPU. Created by @cj_pais, dev work funded by @mozillabuilders. Learn more: localscore.ai/blog
1
7
18
2,181
Llamafile 0.9.2 is out, adding support for DeepSeek-R1, Gemma 3, and IBM Granite! github.com/Mozilla-Ocho/llam…
1
8
21
1,248
llamafile retweeted
Just got back from Mozilla Builders Demo Day. Here's some photos of the @llamafile booth as well as slides from an @openwebui talk via @MattBCool.
8
18
221
23,856
llamafile retweeted
Writing code looks different in the age of genAI. The team at @EclipseSource is working on a fully customizable copilot system for the @Theia_IDE. They’re demoing their work integrating StarCoder2 @llamafile s directly into their editor and it’s 🤯
1
3
6
1,886
llamafile retweeted
I wrote a blog post about all the weird programming language syntax I encountered while building a syntax highlighter for @llamafile. Read justine.lol/lex/
19
44
429
46,433
There's been a lot of progress in #llamafile over the past few months, including improvements to CPU performance and support for new models. We've even created "Whisperfile" which is exactly what it sounds like. Catch up here: hacks.mozilla.org/2024/10/ll…
3
14
1,305
Llamafile 0.8.14 is out, featuring a brand-new command line chat interface github.com/Mozilla-Ocho/llam…
2
17
110
14,121
llamafile retweeted
Can't believe this is a 1B model - it's so fast! Made a video for how to install and get it running for those new to Llamafile. piped.video/Lqh7egmfy4o
2
5
550
Llamafiles are now available for Llama 3.2 1B. Runs great on both GPU and CPU. Just download and go. Bonus: it also includes a sneak peak of our new command-line chat interface! huggingface.co/Mozilla/Llama…
ALT Screenshot of Llamafile running in a terminal, showing the new CLI chatbot interface.
2
2
20
1,692
Llamafiles are also available for the 3B version of Llama 3.2. huggingface.co/Mozilla/Llama…
3
369
llamafile retweeted
Forget ChatGPT: why researchers now run small AIs (local LLMs) on their laptops. My latest for @Nature: nature.com/articles/d41586-0…
Thanks, @SebastienBubeck, @Microsoft, @stlhood, @Mozilla, @sharon000, @drchristhorpe, @embl, @JohnsonThomasMD, @FollowMercy, @cyrilzakka, @Stanford, Kal’tsit, @turboderp_, @eonurkara, @tudelft, @nomic_ai, @llamafile, @ollama, etc.
1
10
42
9,181
llamafile retweeted
We will be hosting a launch party for sqlite-vec on the Mozilla Discord server next week on Tuesday August 6th at 10am PST — come through! discord.gg/4ZMAgmrwqf?event=…
1
8
1,306
llamafile retweeted
🆕 pod: The Winds of AI Winter!
latent.space/p/q2-2024-recap
The vibes have shifted...
@fanahova and @swyx celebrate 1m downloads and recap the last 3 months in AI!
Discussing the Frontier Labs vibe shift between Claude 3.5, Llama 3.1, Apple Intelligence, and the expansion of RAG/Ops into the new war for the LLM OS.
Bonus: @ChatGPTapp voice mode demo with @ethansutin!
1
1
12
26,114
llamafile retweeted
sqlite-vec v0.1.0 is out! Vector search for SQLite that runs everywhere!
- Supported on MacOS, Linux, Windows, browser WASM, Raspberry Pis
- Install with Python, Node.js, Ruby, Go, Rust, Datasette...
- Float and binary vectors supported
Try it out!
alexgarcia.xyz/blog/2024/sql…
10
85
406
31,384
llamafile retweeted
A new blog post: Introducing sqlite-rembed, a SQLite extension for generating text embeddings from remote APIs , all with SQL!
Supported providers include:
- OpenAI
- Nomic
- Cohere
- Jina
- Mixedbread
- Ollama
- llamafile
a sqlite-vec sister project
alexgarcia.xyz/blog/2024/sql…
ALT -- OpenAI embeddings, API key from OPENAI_API_KEY env var INSERT INTO temp.rembed_clients(name, options) VALUES ('text-embedding-3-small','openai'); select rembed('text-embedding-3-small', 'The Mexican Revolution started in 1910...'); -- Jina embeddings, API key from JINA_API_KEY env var INSERT INTO temp.rembed_clients(name, options) VALUES ('jina-embeddings-v2-base-en','jina'); select rembed('jina-embeddings-v2-base-en', 'The Mexican Revolution started in 1910...'); -- Mixedbread.ai embeddings, API key from MIXEDBREAD_API_KEY env var INSERT INTO temp.rembed_clients(name, options) VALUES ('mixedbread-ai/mxbai-embed-large-v1','mixedbread'); select rembed('mixedbread-ai/mxbai-embed-large-v1', 'The Mexican Revolution started in 1910...'); -- Nomic embeddings, API key from NOMIC_API_KEY env var INSERT INTO temp.rembed_clients(name, options) VALUES ('nomic-embed-text-v1.5', 'nomic'); select rembed('nomic-embed-text-v1.5', 'The Mexican Revolution started in 1910...');
3
7
24
1,715
New #llamafile release - 0.8.10 - github.com/Mozilla-Ocho/llam…
Includes the first version of our new "llamafiler" server, with an endpoint that serves up embeddings at 8x the speed of llama.cpp.
2
8
20
1,526
llamafile retweeted
The holy grail of Local AI:
🔓Open Source
🏃♀️Runs Fast
💻Runs on your machine
🌐Runs on all machines
@Mozilla is leading the charge on the new local AI ecosystem: @jartine’s work on @llamafile and @stlhood announced sqlite-vec & $100k in non-dilutive funding for all local AI projects!
Full talk in next post!
4
27
106
23,097
llamafile retweeted
🚀 Resources for Optimizing DevOps with Docker 🚀
1️⃣ Docker Desktop vs. Docker Engine: How to tell which Docker you're running:
bit.ly/4bDPDdd
2️⃣ Simplify AI Development: Containerize @llamafile with Docker:
bit.ly/3S3pAFk
#DockerBlog #DevOps #AI
8
14
6,110