Running AI on your own machine raises two questions at once: which models your laptop can actually run, and which model to use for which task. This first part answers the second one, job by job, across chat, documents, speech, translation, images and video. Every model listed runs through QVAC, the free and open-source toolkit we build for local AI, on laptops and on phones, with nothing leaving your device.
If you have only ever used cloud AI, in ChatGPT or Gemini, moving it onto your own machine raises two questions at once.
This series of article answers them, so you can get started with local AI without guessing.
QVAC is the free, open-source toolkit we build for that, and every model below runs through it, on laptops and on phones.
An AI model is a file. You download it once, and from then on it runs on your own machine. Everything comes from that.
It will not match ChatGPT or Gemini on the hardest problems. On the everyday ones, some of these models are good for their size, and any model you run yourself comes with four things a cloud service cannot offer:
On Hugging Face a model is a list of files, and the file name is the only label you get. It looks like nothing until you know where to cut it.
A file name like Qwen3.5-4B-Instruct-Q4_K_M.gguf carries four facts.
The 3.5 in the Qwen name is the generation: a higher one is more recent, which does not always mean better at your particular job.
Model names carry a number like 4B or 27B. That's the model's size, not its version: B means billion parameters, and more of them usually means better answers and more memory needed. It is not a quality score.
Instruct means the model was tuned to follow instructions and hold a conversation. Other families write it as -it, short for instruction tuned, or -Chat. A base model, usually tagged -Base, continues the text you give it instead of answering it.
Q4_K_M and Q8_0 at the end are quantization levels: how many bits each parameter is stored at. Fewer bits mean a smaller file and less memory.
Learn more about what Quantization is: https://x.com/qvac/status/2060349106346672267
The tables that follow are one per kind of work. "Runs on" is the smallest machine that model is comfortable on, and "Phone" means a phone from roughly the last four years. Text, speech and translation answer in seconds once loaded. Images and music take longer for the media to be generated.
Qwen3.5 4B answers general-knowledge questions, researches a topic and writes usable code, and it fits on a laptop most people already own. Ask it for one thing per request and it holds up. Ask it to research a subject and write a long structured report in a single go and it will hand you something that reads well and is wrong in the middle.
A document reader plus a search engine over your entire archive together download in under a gigabyte, on a phone that came with at least 128 of them.
A language costs 17 to 42 MB, so a dozen of them travel offline in a few hundred megabytes.
Video needs a stronger machine than anything else here. WAN 2.1 needs 20 GB of memory free before it will start, which in practice means a 32 GB machine, and it takes about ten minutes to load before it generates a frame.
Now that you know which model does which job, and roughly what each one needs from a device, you still need to know what your own device can run. The next article in the Local AI 101 series shows you how to check.
QVAC is free and open source: github.com/tetherto/qvac
Searching your own documents with AI means comparing your question against every vector you have stored, and that gets slow and memory-hungry as the collection grows. TurboVec is a vector index that makes the search faster and the stored vectors much smaller, with no training step over your data. It is available in the QVAC SDK, and the RAG code you already write does not change.
Read More
Open-source AI moved fast on the languages that already filled the internet, and left much of the world out of the revolution it started. TranslatePsy-AfriSLM covers 19 African languages, runs on an ordinary phone with no connection, and our smallest model matches or beats systems up to 152 times its size.
Read More
Invoice capture is priced per page and works by uploading your supplier list, your prices, and your bank details. We built a demo (an illustrative example, not a QVAC product) that turns folders of invoices into an accounting table on your own machine: no upload, no per-page bill, and the columns are the ones you decide you need.
Read MoreSearching your own documents with AI means comparing your question against every vector you have stored, and that gets slow and memory-hungry as the collection grows. TurboVec is a vector index that makes the search faster and the stored vectors much smaller, with no training step over your data. It is available in the QVAC SDK, and the RAG code you already write does not change.
Read MoreOpen-source AI moved fast on the languages that already filled the internet, and left much of the world out of the revolution it started. TranslatePsy-AfriSLM covers 19 African languages, runs on an ordinary phone with no connection, and our smallest model matches or beats systems up to 152 times its size.
Read MoreInvoice capture is priced per page and works by uploading your supplier list, your prices, and your bank details. We built a demo (an illustrative example, not a QVAC product) that turns folders of invoices into an accounting table on your own machine: no upload, no per-page bill, and the columns are the ones you decide you need.
Read MoreStay updated
New versions, breaking changes and migration notes - straight to your inbox. No spam, unsubscribe anytime.