Local AI 101: how to deploy local AI in your company

In our Local AI 101 Part 3, we showed when a machine you own costs less than an API or a subscription, and when it's economically viable.

This Part 4 shows how to put local AI to work in a company: which machines to buy, how people reach them, what's the cost for a team of 20, and what to check before everyone relies on it.

Note: the costs below are estimates to illustrate each setup. Do your own calculation for your team before you buy any hardware.

Test: is local AI a good fit for your business?

Answer this quiz to see if local AI is a good match for your company.

With two or more yes answers, local AI is worth setting up.

How you set it up depends on how your company works: one founder, a team that shares an office, or a team spread across cities.

The rest of this article shows how to get started with local AI in each of these three setups.

Local AI setup for your company

👨‍💻 Solo founder

A founder can run everything on one laptop, or add a second machine that keeps agents running when the laptop is closed.

One laptop

A MacBook Pro with 32 GB ($2,399) runs Qwen3.8-27B, which scores 86.0% on SWE-bench Verified, a test built from real bug fixes. At the work-agent volume from part 3, 22 million tokens a month, the laptop costs about $50 a month over four years, against $55 through a mid-range API and $109 through a frontier one. When the laptop sleeps, the agents stop.

A light laptop and an always-on machine

A MacBook Air with 16 GB ($1,299) handles daily work, and a Mac mini with 32 GB ($1,799) stays on the desk and runs agents around the clock, closed laptop or not. Together they cost about $3,100, or about $65 a month over four years. The laptop sends work to the Mac mini through qvac serve on the same network. QVAC's delegated inference, in the @qvac/core package, also reaches the Mac mini from elsewhere: a device paired by invite sends its request over an encrypted peer-to-peer link.

In both setups, a phone paired through delegated inference can use the larger model too, and one hosted API key covers the few problems that need the frontier: the best hosted model scores 97.0% on the same test, 11 points higher.

👥 Team in one office

When everyone works on the same network, the team can share one machine, and requests never leave the building. A Mac Studio with 128 GB ($5,099, about $106 a month over four years) runs larger models than a 32 GB laptop, and a Linux or Windows machine works too: QVAC runs on Ubuntu 22 or later and Windows 10 or later with a GPU driver that supports Vulkan 1.4 or newer. qvac serve exposes the machine as an OpenAI-compatible API, so the coding tools and chat apps the team already uses can point at it by changing their server address.

Running Qwen3.6 35B, a Mac Studio can produce about 200 million tokens in a working month, which covers about nine people at the work-agent volume from part 3, 22 million tokens each.

QVAC can also serve several requests at once on one loaded model, a technique called continuous batching. QVAC builds on the llama.cpp engine, and in llama.cpp's own benchmark 32 requests running together produce about 11 times the output of one, each at about a third of the speed it would get alone. Each request running at the same moment holds its own context in memory on top of the model, so size the machine for the most requests you expect at once, not for the model file alone.

Developers who run a coding agent all day need a 36 GB machine of their own running Qwen3.6 35B, the faster model we presented in Local AI 101 Part 3, because one coding agent uses 110 million tokens a month, more than half of what the shared machine handles one request at a time.

When a model is too big for any single machine, QVAC's cluster inference can split it across several on the office network.

🌍 Remote team

When people work from different cities, each person can run the model on their own machine, so nothing is shared and nothing crosses the internet. Alternatively, the team can share one machine in one city. qvac serve is an ordinary web server, so the rest of the team can reach it through a private network such as Tailscale, which keeps the machine off the public internet and costs from $8 per person a month.

QVAC's delegated inference offers a third route without a VPN: each paired device sends its requests to the shared machine over an encrypted link and the answer streams back, wherever the device is.

Costs for 20 people

20 developers use about 2.2 billion tokens a month. Hosted, that comes to $36,000 to $60,000 a year. A 36 GB machine for each developer costs about $50,000 up front and nothing after that.

2 shared 128 GB Mac Studios cost $10,198 up front, plus $1,920 a year if everyone reaches them through a private network. Together they handle about 400 million tokens a month one request at a time, and more with continuous batching, enough for the everyday agent work of 20 people at 22 million tokens each. 20 coding agents at 110 million each need the 36 GB machines above.

You can also mix local and hosted: run most of the work on your own machines and keep a hosted plan for the few tasks that need a bigger model than they can run.

Go-live checklist

• The shared machine needs access control and an audit log, like any server that holds company data.
• Model weights come with their own license, separate from the code that runs them. Read it before the team starts using the model.
• Keep a small set of real tasks and run it again whenever you change the model or its quantization.
• Plan how models and the SDK get updated, and name the person who owns the machine.

First steps

A founder can start today with the machine they have and the model that fits its memory. A team can start with one shared machine, point the tools it already uses at that machine, and count a month of tokens: the count tells you whether to add a second machine or give the heaviest users 36 GB machines.

If you are thinking of running QVAC in your company, send us a DM on X at @qvac and we will help you set it up.

QVAC is free and open source: github.com/tetherto/qvac

Sources

• Team costs · Claude Code cost documentation; Apple US store, October 2026; Tailscale pricing
• Throughput · llama.cpp batched-bench README
• Coding scores · Vals.ai SWE-bench Verified, September 2026
• Capacity · Local AI 101 part 3, measured with the QVAC SDK on a MacBook Pro M5 Max, 36 GB
• Delegated inference · @qvac/core documentation (npm)
• Continuous batching and system requirements · QVAC documentation, docs.qvac.tether.io