Searching your own documents with AI means comparing your question against every vector you have stored, and that gets slow and memory-hungry as the collection grows. TurboVec is a vector index that makes the search faster and the stored vectors much smaller, with no training step over your data. It is available in the QVAC SDK, and the RAG code you already write does not change.
When you search your own documents with AI, the text is first turned into vectors: long lists of numbers that place similar meanings close together. A search compares your question's vector against the stored ones and returns the nearest. That works well until the collection grows, at which point holding every vector in memory and scanning them all becomes the slow part.
A vector index is what stops that from happening. It keeps vectors compressed so a full scan is smaller and cheaper, then the top hits can be compared more precisely.
This search is the first half of RAG, retrieval-augmented generation: find the passages that answer a question, then hand them to a model as context so it answers from your documents rather than from memory. The index decides how fast that first half runs and how much memory it takes.
TurboVec is one of those indexes, and its particular quality is that it compresses hard without needing to study your data first. The original is an open-source Rust project under MIT, built on TurboQuant, a compression algorithm from Google Research.
QVAC does not depend on that Rust code. We implemented the same TurboQuant-based quantizer natively in the fabric runtime, the forked llama.cpp and ggml layer the SDK's engines already run on, so the algorithm arrives with the SDK and there is no Rust in your dependency tree. We benchmarked ours against the 0.9.0 Rust release we reimplemented: search throughput is close, and the one place ours is slower today is the first ingestion of a collection, which a later version will close.
If the name rings a bell, TurboQuant is the Google Research algorithm QVAC already implements elsewhere in the SDK: there the compression is applied to the KV cache, the memory a model keeps while it generates text. The vector index applies it at the other end, to the vectors your question is compared against. One algorithm, both ends of the same pipeline.
The two act at opposite ends of the same pipeline.
Squeezing vectors down raises one question before any other: does the search still find the right thing? The paper answers it on 100,000 passages with 1,000 queries, scoring TurboVec against an uncompressed exact search over the same data. Those are the authors' measurements of their own implementation, so read them as the properties of the algorithm rather than as timings of the QVAC build.
It finds the right passage. Across those 1,000 queries the correct passage came back inside the top five results every single time, which is exactly what the uncompressed search managed. It came back as the top result 97% of the time, against 100% uncompressed. So what the compression costs is the ordering of the top few results rather than the answer itself, and for a search that feeds an LLM that is the difference that counts: the right passage still lands in the context.
The index is eight times smaller. Each number in a vector is kept in 4 bits rather than 32, so a vector of 1,536 numbers takes 768 bytes where it used to take 6,144. On the paper's 100,000 passages that is 73.2 MB against 585.9 MB, and ten million of those vectors come down from about 61 GB to about 8 GB. This is especially helpful for local AI, where memory is the constraint that decides what runs at all: a phone or a laptop is already sharing its RAM with the model doing the answering.
Query speed. The paper measures two setups:
Most of that gain comes from the architecture rather than from the compression.
One last property matters on a personal device. An index that learns from your data ends up carrying a summary of it, and a summary can be interrogated. The paper got a membership inference attack to guess whether a record was in the collection 57.3% of the time against a learned index, and 50.0% against this one, which is a coin flip. That matters because the collection is somebody's own material: their files, their mail, their notes. Local AI exists so that material never has to leave the device, and an index that quietly encodes a summary of it would put part of it back on the table.
These are the situations where TurboVec shines, and where the performance gain above is largest.
Skip it when the collection is small. For a few hundred chunks, comparing every vector directly is fast, exact and simpler to reason about. Compression would cost you a little accuracy and buy you nothing.
Compressing a vector means storing each of its numbers with fewer bits, which comes down to rounding. Every method has to decide where to put the rounding points.
Product Quantization, the standard approach, works that out from your data: it samples your vectors, groups them, and keeps one representative per group as a reference list (the literature calls it a codebook). The list has to exist before anything can be stored, and it only describes data resembling the sample, so a new language or a new file type compresses worse and can mean building it again.
TurboVec never builds one. Before compressing, it rotates every vector the same way. The rotation is picked at random once, then reused for everything. Rotating keeps the distances between vectors intact, so the search still finds the same neighbours. What changes is the range the individual numbers fall in. After the rotation they spread out in the same predictable way, whatever the data looked like going in. Data that leaned heavily one way and data that sat in a tight clump come out looking much the same.
Knowing the spread in advance means the rounding points can be calculated rather than learned, and the training step goes with them. The paper calls this codebook-oblivious: the compression depends on no property of your data. So new vectors compress exactly like the first ones, and the index holds no summary of your documents.
One flag in qvac.config.json, set before the first SDK call:
{ "ragTurbovec": true }npm install @qvac/[email protected]const modelId = await loadModel({ modelSrc: EMBEDDINGGEMMA_300M_Q8_0 })
await ragIngest({ modelId, workspace: 'my-archive', documents })
const results = await ragSearch({ modelId, workspace: 'my-archive', query, topK: 5 })ragIngest() and ragSearch() keep the same signatures and the same return shapes. What changes is the index underneath. Four things to know:
This is not a separate database you have to run. HyperDB stays the durable store for documents and mutations, the store the SDK's RAG already used, and TurboVec replaces the index on top of it. That is also why the log keeps a [rag:hyperdb] prefix.
To confirm which index a workspace got, read its marker: {"version":1,"adapterType":"turbovec"}.
Almost every constraint that makes local AI hard comes back to memory, or to work you cannot do without a server. TurboVec drops a training pass and shrinks the index eight times, so more of someone's own archive fits on hardware they already own, and none of it has to leave the machine to be searched.
Running AI on your own machine raises two questions at once: which models your laptop can actually run, and which model to use for which task. This first part answers the second one, job by job, across chat, documents, speech, translation, images and video. Every model listed runs through QVAC, the free and open-source toolkit we build for local AI, on laptops and on phones, with nothing leaving your device.
Read More
Open-source AI moved fast on the languages that already filled the internet, and left much of the world out of the revolution it started. TranslatePsy-AfriSLM covers 19 African languages, runs on an ordinary phone with no connection, and our smallest model matches or beats systems up to 152 times its size.
Read More
Invoice capture is priced per page and works by uploading your supplier list, your prices, and your bank details. We built a demo (an illustrative example, not a QVAC product) that turns folders of invoices into an accounting table on your own machine: no upload, no per-page bill, and the columns are the ones you decide you need.
Read MoreRunning AI on your own machine raises two questions at once: which models your laptop can actually run, and which model to use for which task. This first part answers the second one, job by job, across chat, documents, speech, translation, images and video. Every model listed runs through QVAC, the free and open-source toolkit we build for local AI, on laptops and on phones, with nothing leaving your device.
Read MoreOpen-source AI moved fast on the languages that already filled the internet, and left much of the world out of the revolution it started. TranslatePsy-AfriSLM covers 19 African languages, runs on an ordinary phone with no connection, and our smallest model matches or beats systems up to 152 times its size.
Read MoreInvoice capture is priced per page and works by uploading your supplier list, your prices, and your bank details. We built a demo (an illustrative example, not a QVAC product) that turns folders of invoices into an accounting table on your own machine: no upload, no per-page bill, and the columns are the ones you decide you need.
Read MoreStay updated
New versions, breaking changes and migration notes - straight to your inbox. No spam, unsubscribe anytime.