What is a local LLM for construction?

A local LLM is an AI model such as Gemma or Llama that you download and run entirely on your own computer through free software like LM Studio, instead of sending prompts to a provider's servers. It works fully offline, so no project data, contract terms or drawings ever leave your machine.

Is a local LLM more private than Claude or ChatGPT?

Genuinely, yes, because no data leaves your device. But the practical privacy gap with cloud tools is often smaller than people assume: it depends on the specific product's training and retention settings, which you can usually check and often opt out of. Read the actual policy before assuming the worst.

What hardware do you need to run a local LLM well?

Roughly, a 7-billion-parameter model needs about 6 to 7GB of RAM at 8-bit precision, a 32-billion-parameter model needs about 24GB, and a 70-billion-parameter model needs about 42GB. Dedicated graphics memory runs far faster than standard laptop RAM, which is why an ordinary laptop struggles badly.

Does a local LLM save construction businesses money on AI?

Not usually, unless privacy requirements force the decision. The model itself is free, but running it at a usable speed on a standard laptop is painfully slow, and buying hardware fast enough to fix that, like a 128GB Mac Mini around $2,000, often costs more than a year of a frontier-model subscription.

Can a local LLM read construction drawings or do a takeoff?

Most can't. Many open models have no vision capability at all, so uploading a drawing does nothing. Even models built for vision remain unreliable at fine visual measurement, so treat any AI model, local or cloud, as unsuited to owning a takeoff measurement on its own.

When does it actually make sense to run AI locally in construction?

When you're handling genuinely restricted data. Defence-related construction work is the clearest example, where sending anything to an external server isn't an option. Outside that narrow case, a cheap pay-per-token router or a standard subscription gets more reliable output for far less setup effort.

Local AI for Construction: What It Can Actually Do

I’ve been testing local large language models, AI you download and run entirely on your own laptop, completely offline. This comes from a hands-on video where I set one up in under five minutes on my own machine and tested it live. It’s for anyone in construction weighing genuine privacy or cost reasons against just using Claude, ChatGPT or Gemini.

Key takeaways

A local LLM like Gemma or Llama runs entirely on your own machine through free software such as LM Studio, no internet connection required once it’s downloaded.
It’s free beyond your own hardware and electricity, unless you buy dedicated memory to make it usable at real speed.
On a standard laptop it’s painfully slow: roughly 8 words a second on 16GB of ordinary RAM, versus roughly 150 words a second on a 24GB graphics card.
Genuine privacy is the real reason to run one locally, not cost. Cloud privacy policies vary by tool and change over time, so check the current terms before assuming the worst.
My own conclusion after testing: I’m sticking with Claude for real work and keeping an open model for boring admin tasks only.

What is a local LLM, and how is it different from Claude or ChatGPT?

A local LLM is an AI model, like Google’s Gemma or Meta’s Llama, downloaded as a file and run entirely on your own computer through free software such as LM Studio, instead of a cloud model like Claude, ChatGPT or Gemini that runs on a provider’s servers. A local model never sends a token anywhere. You can disconnect your internet connection entirely and it keeps working, because the file that generates the response is sitting on your hard drive, not in a data centre.

Two distinctions matter here, and people mix them up. Local versus hosted is about where the model runs: a local model runs on your machine, a hosted model runs on someone else’s server, even when it’s technically the same open model. Open versus closed is about whether the file itself is public: an open model, like Gemma, Llama, Qwen or the newly released GLM 5.2, can be downloaded and inspected by anyone. A closed model, everything from Claude to ChatGPT to Gemini, keeps its weights hidden. You can still pay a cloud provider to run an open model for you, which is what most people actually end up doing once the model gets too large for a normal laptop.

The other piece people underrate is the harness: the software that loads the model and lets it read a file or run a command. LM Studio is a harness. So is Claude Code, or Cursor, or Ollama if you want to recreate that kind of coding-and-file environment against a local model instead. Swap the harness around the same base model and results change meaningfully. The model matters, but so does what’s wrapped around it.

How do you set up a local LLM for construction in five minutes?

Go to lmstudio.ai, download the free software, open the built-in model search, pick a model such as Gemma, and start chatting, entirely offline once it is downloaded, all in under five minutes on an average laptop with no coding and no account required.

The steps, as I ran them:

Download LM Studio from lmstudio.ai. It’s free.
Open the model search and pick a model. I used Gemma 4, Google’s recently updated open model, about 6GB.
Wait for the download. Gemma 4 took roughly 5 minutes on my connection.
Start a new chat and select the model you downloaded.
Ask your question. I tried “give me an overview of best practices when writing construction inspection and test plans” as the first real test.

You can turn your internet off at that point and the model keeps answering, proof that nothing is leaving your machine. If you want something closer to Claude Code or Cursor, a fuller harness that can read and write files against a local model, look at Ollama instead of LM Studio. I haven’t tested it properly myself, mostly because, as I’ll get to, I don’t think the use case is there yet for construction.

Is a local LLM actually more private than Claude, ChatGPT or Gemini?

A local model is genuinely more private in the strict sense, since nothing ever leaves your device, but the practical privacy gap with Claude, ChatGPT or Gemini is smaller than most people assume, and it depends entirely on the specific product’s settings. Treat “local” as a privacy floor, not an automatic silver bullet.

There are really four things worth checking in any AI privacy policy: whether your data trains future models, whether humans can access and review it, how long it’s retained, and where it’s stored. As of when I recorded this, here’s roughly how the major tools compared:

Tool	Trains on your data by default	Retention (as read at time of recording)
ChatGPT (free tier)	Yes, unless you opt out	Not specified in the policy I checked
Gemini	Yes, on consumer chats	Up to 3 years
Claude Pro / Max	Yes, but you can opt out	Not specified in the policy I checked
Local (LM Studio, Ollama)	No, nothing leaves your machine	N/A, it’s your own hard drive

Policies change constantly, so treat that table as a snapshot and read the current terms for whatever you’re running. Using Google Drive or SharePoint already means storing data on someone else’s servers. What you should actually care about is the training-use and retention answers, not the fact that data touches a cloud at all.

Where local genuinely earns its place is highly restricted work. In Australia, a lot of Department of Defence-related construction work carries privacy requirements strict enough that you can’t send data to an external service at all. That’s the case local AI was actually built for. For a normal commercial job, weigh it against your own comfort with the settings on Claude or Gemini for construction, rather than assuming local is automatically the safer choice.

Does running AI locally actually save you money?

Running AI locally mostly does not save construction businesses money, unless privacy genuinely forces the decision, because the model file itself is free but running it at a speed you would actually tolerate requires hardware that can cost more than a year of a frontier-model subscription. The free part is real. The usable part is where the maths falls apart.

On my own laptop, 16GB of ordinary RAM moving at roughly 15GB a second, Gemma 4 answered at around 8 words a second, closer to watching paint dry than a working chat window. A 24GB dedicated graphics card would run the same model at roughly 150 words a second, fast enough to actually use. That gap is the whole story: a local model has to reload the entire file for every single token it produces, so speed comes down to how fast your machine can move that file, not how smart the model is.

Sizing the hardware to the model follows a rough rule of thumb, in GB of RAM needed at 8-bit precision:

Model size	RAM needed	What it takes to run it
~7 billion parameters	6-7GB	Runs on most modern laptops
~32 billion parameters	~24GB	Needs a dedicated graphics card or unified-memory machine
~70 billion parameters	~42GB	Needs serious hardware, e.g. a 128GB Mac Mini around $2,000

That $2,000 machine buys you a model that’s still a long way behind Claude, ChatGPT or Gemini on most real tasks. If cost is genuinely the driver, a cheaper route is a pay-per-token router like OpenRouter or Groq, which hosts open models on someone else’s hardware and charges close to raw compute cost, no frontier-model margin, no capital outlay on your end. The skills built and shared inside the ContractorOS community assume a capable model doing the reasoning step; running the same workflow against a smaller open model would mean rebuilding the guardrails yourself, and testing it a lot harder before trusting the output.

What can a local model actually do for construction, honestly?

Honestly, a local open model cannot do much for construction yet, beyond situations with genuinely restricted data, because even a capable-looking model gave generic, non-construction-specific answers on my own test, and most open models cannot read a drawing at all. That’s the read after actually using one, not the read from a spec sheet.

Gemma 4 answered my inspection-and-test-plan prompt with a generic structure: understand the goal, determine the audience, structure the guide, flesh out each section. That could apply to any industry. Nothing construction-specific, nothing I’d hand to a project engineer. I also tried Microsoft’s local model, Phi, on a simple question about how local LLMs work, and it visibly confused itself mid-answer and never properly recovered.

Some open models genuinely compete with the frontier now. GLM 5.2 has reportedly matched Opus 4.8-level results on benchmarks, but most open models, GLM 5.2 included, have no vision capability at all. Upload a construction drawing to GLM 5.2 and it does nothing with it. If you’re weighing that gap against a proper model comparison for daily use, Claude vs ChatGPT for construction covers where the frontier tools actually differ on real tasks.

As I said on camera: “I’m a Claude lover. I use Claude for everything.” That’s not brand loyalty, it’s built-up context and trust in what the tool can and can’t do, earned over time. My actual plan for open models isn’t running them locally at all. It’s using one through an agent for boring, repetitive admin work, and leaving Claude for anything that matters: “The way I’m actually planning on using them is I want to use GLM 5.2 within my Hermes agent, which is what I’m going to use for more boring repetitive admin tasks and leave Claude to the more important stuff.”

Common mistakes and what to watch for

Assuming “free” means free. A model file costs nothing, but a machine fast enough to run it well can cost $2,000 or more, often more than a year of a frontier-model subscription.
Assuming local automatically means safer. Read the actual privacy policy of the cloud tool you’re comparing it against. The gap is often smaller than the reputation.
Testing on an average laptop and blaming the model. Eight words a second is a hardware bottleneck, not a verdict on the model’s intelligence.
Expecting vision where there isn’t any. Most open models can’t read a drawing at all. Don’t hand one a plan set and expect anything useful back.
Treating a demo answer as proof of readiness. One generic response to an inspection-and-test-plan prompt isn’t a pilot. Test it against real project documents before trusting it with anything.

None of this rules local models out long-term. The more interesting effect is what open models do to pricing: they act as a ceiling on what Claude, OpenAI and Gemini can charge, because a real cheap alternative now sits underneath them. That’s a better reason to watch this space than to expect you’ll be running your own model any time soon. This sits inside the wider question of AI for construction: where it earns its keep, and where the honest answer is still “not yet.” I walk through the LM Studio setup and the hardware maths live in the video above.