The engine layer
QVAC Fabric
Fabric is the layer where inference and fine-tuning happen, on the device. It is what QVAC runs on, from a phone to a workstation.
import { loadModel, finetune, QWEN3_600M_INST_Q4 }
from "@qvac/sdk"
const modelId = await loadModel({
modelSrc: QWEN3_600M_INST_Q4,
modelType: "llamacpp-completion"
})
const handle = finetune({
modelId,
options: { trainDatasetDir: "./data",
outputParametersDir: "./out" }
})
await handle.result // a .gguf LoRA adapter in ./outOne engine
Inference and fine-tuning on the same layer
Both share one engine and one set of GPU backends. A device that can run a model can also adapt one.
Inference
Loads a model and answers, using whatever GPU the machine has. Supports BitNet 1-bit models alongside standard quantized formats.
Fine-tuning
Trains a small LoRA adapter on your own examples, so the model learns your domain. The adapter comes back as a file you can ship and load anywhere.
Mobile GPUs
Running inside mobile memory limits
Mobile GPUs enforce hard memory limits, and a standard training run exceeds them and fails. Fabric works within those limits.
Dynamic tiling
Large matrix operations are split into tiles that fit within constraints such as the Adreno 128 MiB SSBO limit, which keeps training stable on a phone.
No CUDA requirement
Vulkan and Metal backends cover AMD, Intel, NVIDIA, Apple Silicon, Adreno and Mali, so the same code runs from a phone to a server.
Quality holds up
Near-parity with established frameworks on our benchmarks.
0-94%
accuracy on biomedical question answering,
with win rates comparable to PyTorch
Private by design
Personalization that never leaves the device
A base model is generic. Fine-tuning makes it yours. Fabric trains a LoRA adapter on your own data, on the device, so the model learns your domain without sending a single example to a server.
Your data stays local
Training runs on-device, nothing is uploaded.
GPU-accelerated
Vulkan and Metal, desktop and mobile, even a phone.
Portable output
A small .gguf adapter you can ship and load anywhere.
FAQ
Questions developers ask
Where it comes from
Fabric is a fork of llama.cpp. llama.cpp already supports training, but historically on CPU only, with no LoRA fine-tuning and no instruction-tuning logic. Fabric adds GPU-accelerated LoRA, masked-loss instruction tuning, and the mobile GPU work that makes both run on a phone.
What hardware can I fine-tune on?
Most modern consumer hardware, without CUDA. The Vulkan and Metal backends cover Android with Adreno or Mali, iOS and macOS on Apple Silicon, and Windows or Linux on AMD, Intel and NVIDIA.
How does training survive a mobile GPU?
Mobile GPUs impose strict memory limits, such as the 128 MiB SSBO cap on Adreno, and a standard training run hits them and fails. A dynamic tiling algorithm breaks large matrix operations into smaller tiles, which keeps training on a model like Qwen3 stable on a phone.
What is instruction fine-tuning?
Training a model on target responses, so it follows the instruction it was given.
What is masked-loss training?
The mechanism that makes instruction fine-tuning work. A mask means the model learns only from assistant responses, while user and system messages still provide context and are excluded from the loss. Without it, the model learns to reproduce the prompt format.
Does my data leave the device?
No. Fine-tuning runs locally, with no cloud egress and no API keys. That matters for healthcare and enterprise work where the dataset cannot leave your control.
How does quality compare to PyTorch?
Our benchmarks show near-parity. On biomedical question answering, models fine-tuned with Fabric reached 79 to 94 percent accuracy, with win rates comparable to the same work done in PyTorch.
Get started
Run AI on what you own
One install, JavaScript or Python. 10+ AI tasks. Private, offline and free on the hardware you already own.