Local AI vs cloud AI: what actually changes
We sell the local option, so the second half of this post — the things running a model on your own machine does not fix — is the half worth reading.
Local AI means the language model runs on your own processor or graphics card, from weights stored on your disk. Cloud AI means your prompt, and anything you attach to it, travels to a provider's servers, which run the model and send an answer back.
Everything else people argue about follows from that one difference. Here is what genuinely follows, and what does not.
Four things that change
1. Your files stop being uploads
This is the whole argument, and it is not really about trust in any particular provider. It is about surface area. A document that never leaves your disk cannot be retained past a policy change, cannot be exposed by a breach at a company you have no relationship with, and cannot be caught by a subpoena aimed at someone else. For anything under NDA, under HIPAA, under legal privilege, or simply under a colleague's expectation of discretion, that is a different category of risk rather than a smaller amount of it.
It also removes an argument. “Can I paste this into an AI?” stops needing an answer when the AI is a process on your own machine.
2. Cost stops compounding
Cloud AI is priced per seat per month, forever, and the price is set by whoever is running the datacenter. Local AI has a hardware cost you have probably already paid and, in our case, a one-off licence. The interesting difference is not the total — over two years the numbers can land close — it is that one of them is a decision you make once and the other is a decision someone else keeps making for you.
3. It works with no connection
On a plane, on a train, on a client site with a locked-down guest network, on an air-gapped machine. Once the models are downloaded there is no service to be unreachable, no status page, and no rate limit. Your GPU is the limit.
4. Nobody can deprecate your model
A cloud model can change under you: a new version reasons differently, an old one is retired, a safety filter tightens, a price tier moves. Weights on your disk behave the same way in a year as they do today. If you have built a workflow around a model's particular behaviour, that stability is worth something real.
Three things that do not change in your favour
1. Raw capability
A model that fits on a consumer graphics card is smaller than the largest cloud models, and it shows on hard reasoning, unusual knowledge and complex multi-step code. We are not going to pretend otherwise, and you should be suspicious of any local AI vendor who does. If your work is “solve this genuinely difficult problem”, the frontier models are still better at it.
What local models have become very good at is the work most people actually do with an assistant: reading documents you supply, extracting and summarising, drafting from material you provide, restructuring, and answering questions where the answer is in the files rather than in the model's memory. Retrieval narrows the gap, because the hard part becomes finding the right passage rather than recalling the world.
2. Very long context
Context length is memory, and memory is the constraint. Cloud providers can throw a rack at a million-token window. On an 8 GB card the KV cache for a long context competes directly with the weights. Local tools work around this with retrieval — indexing your files and pulling in the relevant parts — which is often better practice anyway, but it is a workaround, not parity.
3. “Local” by itself does not mean private
Local inference and a private application are different claims. An app can run the model on your machine and still send telemetry, sync your conversations, or phone home with usage analytics. The question to ask a vendor is not “does it run locally” but “what exactly does it send, to whom, and when”. We wrote out our own complete answer, including the parts that do use the network, because a list you can check is worth more than an adjective.
Side by side
| Cloud AI | Local AI | |
|---|---|---|
| Where the model runs | Provider's servers | Your CPU or GPU |
| Your files | Uploaded to be read | Read in place, never sent |
| Cost shape | Per seat, per month | Hardware you own, plus a one-off licence |
| Works offline | No | Yes, after setup |
| Rate limits | Yes | Your hardware is the limit |
| Peak reasoning | Strongest available | Good, not frontier |
| Context window | Very large | Bounded by memory; retrieval compensates |
| Model stability | Changes when the provider changes it | Fixed until you replace it |
| Setup | Sign in | Download a runtime and models once |
Which to choose
Choose local when the material is confidential, regulated or privileged; when the work is mostly about documents already on your machine; when you need it offline or on a restricted network; when you want a predictable one-off cost; or when you simply do not want a third party in the loop.
Choose cloud when you need the strongest available reasoning on a hard problem, when you need a very large context window in one pass, when you need a capability no local model has, or when you do not own hardware you can dedicate to it.
Look at the last ten things you asked an AI to do. If seven of them involved a file that was already on your computer, the local option covers most of your week — and the other three are worth a cloud subscription only if they are actually hard.
The boring answer: both
Most people who use local AI seriously end up running both. The local assistant is the default because it is right there, it is fast on short work, it reads your files without an upload dialogue, and it costs nothing per message. The cloud model is the thing you escalate to a few times a week.
That split has an underrated property: it moves the majority of your AI usage — and therefore the majority of your documents — off the network entirely, while keeping the frontier available for the cases that need it. You do not have to pick a side to get most of the benefit of the private one.
Lemonade is a local AI assistant for Windows and Linux. $49 once, 14-day free trial, no account. Questions: support@aionx.aionapp.org.