Garlic LLM: Developer Runs Offline AI Chatbot on Original PS4
PlayStation 4 Can Now Run a 9B AI Model on Its GCN GPU at Up to 39 Tokens Per Second
The original PlayStation 4 can now run a 9-billion-parameter AI language model entirely offline. Developer Leonardo Alegría built an open-source application called Garlic LLM that runs Qwen3.5-9B directly on Sony's 2013 console, using its AMD GCN GPU to generate text at 31 to 39 tokens per second — no Linux, no external computer, no internet connection required. It's one of the most impressive homebrew feats on the aging console to date.
The project overcomes hardware limitations that should have made this impossible, from the PS4's lack of GPU compute APIs to its modest 8GB of shared memory. Here's exactly how Garlic LLM works, what it can actually do, and why it matters beyond novelty.
Can the PlayStation 4 run an AI model offline?
Yes. Garlic LLM runs Qwen3.5-9B entirely on the PS4's own hardware, achieving roughly 31 to 39 tokens per second in standard chat use and up to 43 tokens per second when generating code. The application is fully open-source and operates without Linux, a PC, or any network dependency — the console generates AI responses completely on its own.
For context, 39 tokens per second is fast enough for comfortable interactive use. Most people read comfortably at a similar pace, so responses feel near-instant rather than laggy. That puts the PS4 in surprisingly practical territory for a machine released in 2013 — a year when large language models weren't even on the consumer radar.
How did the developer get AI running on PS4 hardware?
Alegría bypassed the PS4's missing compute infrastructure by writing custom GPU kernels from scratch. The console was never designed for machine learning, and its homebrew environment offers no dedicated GPU compute API or ML libraries. Instead of waiting for those tools to exist, the developer built them himself.
Custom OpenCL kernels and PM4 command buffers
The technical approach breaks down into three key pieces:
- OpenCL C kernels compiled for AMD's GCN architecture. These hand-written kernels handle the mathematical heavy lifting of the language model on the PS4's GPU, which is based on AMD's Graphics Core Next design.
- PM4 command buffers written by hand. Rather than routing work through a conventional graphics driver, Garlic LLM communicates with the GPU directly using AMD's native command packet format. This eliminates driver overhead the homebrew ecosystem can't optimize around.
- Custom memory management. With no ML framework handling allocations, the developer had to carefully orchestrate how the model's data moves through the console's unified 8GB GDDR5 memory pool.
Each of these would be a substantial project on its own. Combined, they turn a gaming console with zero AI tooling into a functioning local inference machine.
How does a 9B model fit in the PS4's 8GB of memory?
Custom quantization squeezes Qwen3.5-9B into approximately 4.5 GiB of the PS4's shared memory. The technique reduces the precision of the model's stored weights, trading a small amount of accuracy for massive space savings.
The quantization scheme is applied selectively for the best balance of quality and size:
- Most model weights use 4-bit precision, cutting memory usage per parameter by roughly 75% compared to standard 16-bit storage.
- The larger MLP gate and up-projection matrices drop to 3-bit quantization, saving additional space on the model's biggest internal components.
- The KV cache also uses 4-bit precision, which allows Garlic LLM to support a 16K-token context window — the amount of conversation and text the model can consider at once.
This selective approach matters because not all parts of a language model degrade equally under quantization. By keeping critical components at higher precision and compressing the rest, Alegría preserved usable output quality within severe memory constraints.
What is speculative decoding and why does it matter here?
Speculative decoding nearly doubled Garlic LLM's speed, pushing generation from about 22 tokens per second to 31–39. The technique lets the model predict multiple tokens ahead and verify them together in a single pass, rather than generating one token at a time sequentially.
Garlic LLM implements this using Qwen3.5's built-in Multi-Token Prediction (MTP) head, which can propose up to two tokens in advance. When the predictions are correct — which happens frequently in natural language — the system effectively generates three tokens for the computational cost of verifying one pass. When they're wrong, the model simply falls back to standard single-token generation, so there's no quality penalty.
On modern AI accelerators with enormous memory bandwidth, speculative decoding is a nice optimization. On a 2013 console GPU with a fraction of that bandwidth, it's the difference between a sluggish demo and a genuinely usable chatbot.
What can Garlic LLM actually do on a PS4?
It's best understood as a functional local chatbot with real limitations. The honest capabilities breakdown looks like this:
- Basic conversation works well. General chat, explanations, and tool calling are within reach of a 9B model at these speeds.
- Code generation is possible but limited. The developer reports up to 43 tokens per second on code output, though this particular Qwen model isn't especially strong at coding tasks.
- The 16K context window is restrictive by 2026 standards. Long documents or extended multi-turn conversations will hit the limit quickly compared to modern models supporting far larger contexts.
- Everything runs offline with voice support. The application includes a voice mode, making the PS4 a self-contained AI assistant that never phones home.
Nobody is replacing their cloud AI subscription with a jailbroken PS4. But as a demonstration of what's possible with clever engineering — and as a privacy-friendly offline tool — the project punches far above its hardware class.
Why does running AI on old consoles matter?
Projects like Garlic LLM show that local AI isn't reserved for expensive modern hardware. The PS4 sold over 117 million units worldwide, and millions of those consoles are still sitting in homes years after their gaming prime. Demonstrating that this installed base can run useful AI models has implications beyond one developer's hobby project.
There are a few angles worth considering:
- E-waste reduction. Repurposing aging consoles as local AI devices gives old hardware a meaningful second life.
- Offline AI access. In regions with unreliable internet or privacy concerns about cloud AI, offline inference on hardware people already own is genuinely valuable.
- Homebrew innovation. The project pushes the PS4 homebrew ecosystem forward, creating tools and techniques other developers can build on.
- Efficiency research. Squeezing performance out of constrained hardware often produces optimizations that benefit more powerful systems too.
The broader trend is clear: as models get smaller and quantization techniques improve, the minimum hardware bar for running AI keeps dropping. A console from 2013 joining that list is a striking milestone.
Frequently Asked Questions
Can a PS4 really run a large language model?
Yes. Garlic LLM runs the 9-billion-parameter Qwen3.5-9B model fully offline on the original PS4, generating text at 31 to 39 tokens per second using custom GPU kernels written for the console's AMD GCN architecture.
How fast is AI on the PlayStation 4?
Garlic LLM achieves approximately 31 to 39 tokens per second in standard chat use, reaching up to 43 tokens per second when generating code. That's fast enough for comfortable interactive conversation.
Does Garlic LLM require internet or a PC?
No. The application runs entirely on the PS4 itself — no Linux installation, no external computer, and no internet connection are needed. All inference happens locally on the console's GPU.
How does the 9B model fit in the PS4's memory?
Custom quantization compresses most weights to 4-bit precision, with the largest matrices at 3-bit and the KV cache at 4-bit, fitting the model into roughly 4.5 GiB of the PS4's 8GB shared GDDR5 memory.
Is Garlic LLM open source?
Yes. Garlic LLM is an open-source project developed by Leonardo Alegría, who shared the details publicly so other developers can study, use, and build upon the work.
Can the PS4 run bigger AI models?
The 8GB memory ceiling makes significantly larger models impractical on original PS4 hardware. The PS4 Pro's additional memory and GPU horsepower could offer more headroom, but that remains unexplored territory for now.
Key takeaways
- The 2013 PlayStation 4 runs a 9B AI model offline at 31–39 tokens per second thanks to Garlic LLM, an open-source homebrew project by Leonardo Alegría.
- Custom OpenCL kernels and hand-written PM4 command buffers bypass the PS4's lack of GPU compute APIs and ML libraries entirely.
- Selective 3-bit and 4-bit quantization compresses the model into 4.5 GiB, fitting comfortably within the console's 8GB shared memory while preserving a 16K-token context window.
- Speculative decoding via the model's MTP head nearly doubled speed, from roughly 22 to 31–39 tokens per second.
- The project has real limitations — small context window, modest coding ability — but stands as remarkable proof that local AI can run on hardware over a decade old.
Garlic LLM won't replace your cloud AI tools, and it isn't trying to. What it proves is more fundamental: with enough engineering creativity, even a 2013 game console can host a working offline AI assistant. As quantization and inference techniques continue to improve, the list of devices capable of running meaningful AI will only grow. If you found this breakdown interesting, share it with a fellow retro-tech or AI enthusiast — and keep an eye on what the homebrew community does next with this codebase.