My Wi-Fi cut out on a Tuesday afternoon while I was turning a 42-minute meeting transcript into action items. Claude: dead tab. ChatGPT: spinner. Fifteen seconds later, Gemma 4 was chewing through the same transcript on my laptop with the airplane-mode icon staring at me from the menu bar. No cloud, no API key, just a structured list of owners and deadlines from a model sitting on my SSD. That was the moment local AI stopped being a hobby for me and became infrastructure, and the reason it took fifteen seconds instead of an evening of terminal wrestling is LM Studio: a desktop app where the entire setup is click download, click load, start typing.
I have been running this stack for a couple of weeks across a MacBook Pro and a mid-range Windows PC, plus an older travel laptop. It has not replaced the cloud models, and I will show you exactly where they still win. But a surprisingly large slice of my daily work has quietly moved local.
Why this is worth an afternoon in 2026
Three cracks in the cloud-only deal widened this year. Cost: I was running roughly $180 a month across Claude Pro, ChatGPT Plus, and a Cursor seat, plus API credits that could eat $20 in an afternoon when an agent loop went sideways. Privacy: every prompt is a document leaving your machine, which is fine for most work and genuinely not fine for client contracts or half-finished code. Availability: APIs go down, rate limits land at the worst moment, and Wi-Fi dies mid-transcript, as mine did.
Gemma 4 matters because it is the first open model where running locally does not feel like a compromise. Google released it on April 2, 2026 under Apache 2.0: genuinely open, commercially usable. I had already been down this road with Claude Code pointed at free local models through Ollama, so my bar was set by experience, not marketing. Gemma 4 through LM Studio cleared it.
The four sizes, and why I run the 4B most days
Gemma 4 ships in four variants, and picking the wrong one is the single most common mistake I see. It is the difference between "this is fast" and "why is my fan screaming."
| Model | Params | Active per token | Context | Where it runs |
|---|---|---|---|---|
| E2B | ~2B | ~2B | 128K | Old laptops, low-RAM machines |
| E4B | ~4B | ~4B | 128K | Most mid-range laptops and desktops |
| 26B MoE | 26B | ~3.8B | 256K | 32GB+ RAM machines |
| 31B Dense | 31B | 31B | 256K | High-VRAM GPUs, workstations |
Start with the 4B. It delivers roughly 90 percent of what the 26B gives you on common tasks (summarization, structured extraction, Q&A, moderate coding help) at a fraction of the footprint. On my MacBook Pro (M3 Pro, 18GB unified memory) it streams at 45 to 60 tokens per second, fast enough that I forget it is not the cloud.
The 26B MoE is the interesting one if you have the RAM: only about 3.8 billion parameters activate per token, so it runs far faster than a dense 26B would, and reasoning quality visibly jumps. But it wants 32GB minimum, and if you do not have it, LM Studio spills to disk and grinds. The E2B runs on my older Windows travel laptop and is honestly fine for "turn this wall of text into bullets," weak on real reasoning. The 31B dense is for people with 24GB+ VRAM cards, and if that is you, you already know.
Check your RAM before you pick: 8GB means E2B, 16GB means 4B, 32GB and up means try the MoE. LM Studio will happily let you load a model too big for your machine, and it will be miserable.
Install in five minutes
Download LM Studio from lmstudio.ai (Mac, Windows, Linux; roughly a 500MB one-time download). When first launch offers developer mode, say no for now; it exposes the local API server and advanced settings you do not need for chat.
Open model search (Cmd/Ctrl+K) and type gemma 4. Two format notes decode the intimidating names. GGUF is the llama.cpp format and works everywhere; MLX is Apple's framework and is faster on Apple Silicon, so prefer it on M-series Macs when available. The -it suffix means instruction-tuned: always pick it for chat, because base models are for fine-tuning and feel weirdly non-conversational.
Then the quantization selector. Q4_K_M is the right default for almost everyone: the accepted community sweet spot, and I have run side-by-side tests against Q8_0 where I genuinely could not tell the difference on real tasks. The 4B at Q4_K_M is about a 2.5GB download.
Load it from the chat view's model selector. On my MacBook the 4B takes about 3.2GB of RAM. Say yes to GPU offloading; LM Studio auto-detects your GPU and fits as many layers as it can, splitting the rest to CPU on bigger models. That auto-split is where the app earns its keep versus hand-tuned terminal setups.
The three tests I ran before trusting it
Benchmarks are one thing; surviving my Tuesday is another.
Meeting transcript to action items. A real 2,800-word client-call transcript, four participants, tangents and half-decisions. My daily extraction prompt: owner, deadline or "none stated," one-sentence context, markdown table. Gemma 4 4B produced a clean seven-row table with owners correctly attributed, five runs on five different transcripts, structurally right every time. The one miss: a hedged "maybe get Priya involved by Q2" became Priya as owner, a nuance the frontier cloud models caught on the same transcript. For the 90 percent of meeting-notes work that is mechanical extraction, the 4B is fully sufficient.
The differentiator I almost missed: LM Studio's think-mode toggle (the reasoning icon in the chat input). It runs a multi-step reasoning pass before answering, two to three times slower, noticeably better on multi-step inference. My settled rule: skip it for extraction, enable it for "figure out what these four people are actually disagreeing about."
Whiteboard photo to structured notes. Gemma 4 is multimodal natively. I dragged a badly lit photo of my own terrible whiteboard handwriting into the chat and asked for a shareable summary. It correctly read a scrawled flowchart as three-stage onboarding with a branching decision at step two. It misread one abbreviation ("CR" as Customer Relations rather than Code Review), a context call it could not have made. One caveat that will save you a confused hour: not every quantization includes the vision encoder. Some community re-quants strip it to save space, so confirm the model card says multimodal before downloading.
Code review on a real PR. A 340-line TypeScript PR from one of my Next.js projects, prompt: flag bugs, security issues, architecture concerns, be direct. Gemma 4 caught four real issues, including a missing input validation on an API route accepting user-supplied IDs. It missed a subtle async race condition and a type-narrowing issue that Claude traced correctly through three files on the same PR. Honest verdict: competent for day-to-day review, measurably behind the frontier on cross-file reasoning. That is expected from the size difference; what is new is "competent for 80 percent of asks" running free on a laptop.
The LM Studio features that make it a daily driver
Most tutorials stop at the install. These four features are why I stayed.
Branching. From any assistant message, the three-dot menu lets you fork a new thread that keeps the full context. When I want to try three angles from the same setup, each becomes its own branch, nested under the parent in the sidebar. Killer feature, barely mentioned anywhere.
Folders. I file every chat into Work, Writing, Code, or Experiments, and a month in I can find anything in seconds.
Split view. Same prompt to the 4B and the 26B side by side, watching both stream, is the fastest way to learn what the extra parameters actually buy you for your tasks.
Per-chat system prompts. My code-review chats carry "senior engineer, bugs first, style second, complete code blocks only." My writing chats carry "bullets only, no preamble." Set once, saved forever, plus a global default in settings.
One more habit: when a chat drifts, do not start over. Edit the message where the drift began and regenerate from there. The context stays clean and the model recovers.
Where the cloud still wins
Local honesty requires this section. Long-context reasoning degrades noticeably past about 32K tokens of input even though the spec sheet says 128K or 256K; for whole-codebase analysis, the million-token cloud models are a different sport, with their own discipline I covered in managing 1M-token context. Deep multi-file coding reasoning still belongs to the frontier. There is no web search and a fixed knowledge cutoff, so anything requiring fresh data needs a cloud model or a retrieval stack. And while Gemma 4 supports function calling natively, complex agent loops are still smoother in the mature cloud ecosystems. If your interest is specifically open-weights agents at the heavy end, my Kimi K2.6 test covers what a trillion-parameter open model does that a 4B cannot.
My working split after two weeks: roughly 70 percent of what I used to send to the cloud now runs locally, and the remaining 30 percent is exactly the deep-reasoning work the subscriptions still earn. That 70 percent running free, offline, and private is a genuinely new category.
Common questions
What are the minimum requirements?
The 4B at Q4_K_M wants 16GB RAM and about 3GB of disk. The E2B runs on 8GB machines. The 26B MoE wants 32GB RAM minimum. Apple Silicon or any modern GPU helps but is not required.
Is this stack legal for commercial work?
Yes. Gemma 4 is Apache 2.0, which explicitly permits commercial use, and LM Studio is free for personal and commercial use as of this writing. Check current terms for edge cases.
Can it run fully offline?
Completely. Once the model file is on disk, inference makes zero network calls. Airplane mode is fine. That is the entire point.
Your next ten minutes: download LM Studio, install Gemma 4 4B at Q4_K_M, and paste in one real document from today's work. You will know within the first response whether this belongs in your stack. I knew during the Wi-Fi outage, and when the connection came back, I left Gemma running anyway. If you handle sensitive client data and want a private local AI environment specced correctly for your team's actual hardware, that is a setup I build for people; tell me your machines and your workload and I will tell you which model sizes will actually run well.
