Fabric, formally qvac-fabric-llm.cpp, is QVAC’s maintained fork of llama.cpp. It keeps compatibility with upstream llama.cpp while adding QVAC-specific capabilities such as TurboQuant KV-cache quantization, mobile GPU optimizations, BitNet support, and native LoRA fine-tuning across CPU, Vulkan, and Metal.
Inside QVAC, Fabric is the native engine that powers @qvac/llm-llamacpp. It is the layer that executes LLM inference and fine-tuning, while @qvac/llm-llamacpp wraps that engine with the JavaScript/Bare addon API, model lifecycle behavior, cache management, context sliding and other features used by the SDK.
The problem addressed here is specific to M-RoPE/iM-RoPE models such as Qwen3.5 style decoders. These models use multi-axis positional encoding for text and multimodal tokens, so the normal scalar KV-cache shift used for standard RoPE models is not enough.
We needed that shift support because QVAC applications keep interactive sessions alive across many turns. A chat can accumulate system prompts, tool definitions, tool results, user messages, assistant replies, and, for Qwen3.5 style models, image-derived tokens. Eventually the active context approaches the model’s context limit.
Without context shifting, the SDK has only bad options when the prompt no longer fits: fail with context overflow, clear the session and lose continuity, or reprocess a compacted prompt from scratch. Sliding context is the cheaper and more user-friendly option. It removes older tokens from the KV cache and keeps generation moving while preserving recent conversation state.
What was the problem in Fabric?
Standard RoPE models have one logical token position per KV-cache entry. When the context window slides, Fabric can remove old KV cells and apply a scalar position shift to the remaining cells.
M-RoPE/iM-RoPE models are different. Qwen3.5 style models encode positions across multiple axes, such as temporal, spatial-y, spatial-x, and an additional/unused axis depending on the model layout. A single scalar shift is not enough.
The old Fabric behavior had several failure modes:
- seq_add() assumed n_pos_per_embd() == 1, so attempting to slide multi-position models could assert or abort.
- K-shift uses one scalar shift plane, but M-RoPE needs multi-axis shift data.
- Shifted image tokens could lose their spatial/temporal layout.
- Quantized K-cache shifting needed explicit handling for q8, TurboQuant, and PolarQuant CPU layouts.
In practice, this meant sliding context for Qwen3.5 could either fail outright or rotate cached K tensors with the wrong positional data.
Was this only a Fabric problem?
This was not only a local Fabric limitation. Upstream llama.cpp also treated M-RoPE KV-cache shifting as unsupported. Current upstream still guards shifting for multi-position embeddings:
if (hparams.n_pos_per_embd() > 1) {
return false;
}and seq_add() / seq_div() still assert n_pos_per_embd() == 1.
So Fabric was not fixing a purely local QVAC bug. It was implementing support that upstream either did not support yet or was explicitly disabled for M-RoPE K-shift.
How the problem was fixed in Fabric
The K-shift path in Fabric changed from “move every cached token by one scalar position” to “move each cached token by the same kind of multi-axis position delta that was used when the token was evaluated”.
At a high level, a context slide still starts the same way. The caller removes a range of old KV cells, then shifts the remaining cells left so their positions are contiguous again. The difference is what gets recorded while doing that shift.
For a standard RoPE model, the shift path only needs to remember one number per shifted KV cell:
token at position 800 slides left by 256
pos = 800 - 256
shift = -256For an M-RoPE/iM-RoPE model, the cached token also has multi-axis position metadata. Text tokens and image tokens both participate in the decoder stream, but image tokens carry spatial grid coordinates. If only pos is updated, the cache metadata says one thing while the already-rotated K tensor still reflects another. The fix was to replace the scalar cell shift with a structured shift delta:
struct llama_kv_cell_shift {
llama_pos t = 0;
llama_pos y = 0;
llama_pos x = 0;
llama_pos other = 0;
};The actual cell mutation is implemented through pos_add() and pos_shift(). For scalar RoPE, pos_add() only fills delta.t. For M-RoPE/iM-RoPE, the shift_ext path also fills the spatial axes before delegating to pos_shift():
bool pos_add(uint32_t i, llama_pos d, bool shift_ext = false) {
llama_kv_cell_shift delta;
delta.t = d;
if (shift_ext) {
delta.y = d;
delta.x = d;
}
return pos_shift(i, delta);
}
bool pos_shift(uint32_t i, const llama_kv_cell_shift & d) {
pos[i] += d.t;
ext[i].y += d.y;
ext[i].x += d.x;
shift[i].t += d.t;
shift[i].y += d.y;
shift[i].x += d.x;
shift[i].other += d.other;
}That pending shift is later consumed by the K-shift graph. This is the part that actually re-rotates the cached K tensor so the data in memory matches the new positions.
Before this change, the K-shift input was shaped like one scalar shift per KV cell. In Fabric, this was changed for M-RoPE/iM-RoPE so the graph receives four shift planes:
standard RoPE:
k_shift[cell] = delta
M-RoPE/iM-RoPE:
k_shift[t][cell] = delta_t
k_shift[y][cell] = delta_y
k_shift[x][cell] = delta_x
k_shift[other][cell] = delta_otherWith that input available, the K-shift graph can use ggml_rope_multi / ggml_rope_multi_inplace and pass the model’s rope sections and rope type into the graph. That matters because M-RoPE does not rotate the whole vector as one uniform scalar-position RoPE space. The tensor must be re-rotated according to the same axis partitioning that the model used during the original forward pass.
The implementation also tightened image-token handling. Image chunks are shifted by moving their origin while preserving relative grid offsets, so an image patch grid remains a grid after context sliding. Still-image chunks that contain varying temporal positions are rejected, because that would make the shift ambiguous for this representation.
Quantized K-cache support needed a separate pass because q8, TurboQuant, and PolarQuant do not all look like plain contiguous floating-point K tensors on CPU. Coverage was added for those CPU layouts so the K-shift path can correctly copy, rotate, and write back quantized K-cache rows.
Finally, seq_div() was guarded for M-RoPE. Dividing positions by an integer is sometimes used for cache manipulation in scalar-position models, but applying integer division to image-grid coordinates can collapse distinct spatial positions into the same coordinate. For M-RoPE, rejecting that operation is safer than silently corrupting the grid.
Summary
The native engine can now shift M-RoPE/iM-RoPE KV cache correctly.
@qvac/llm-llamacpp then used that capability safely for long-running text and multimodal Qwen3.5 sessions.
Together, they turned a previously unsafe or unsupported operation into a tested sliding-context path that preserves image-token layout, cache save/load behavior and quantized K-cache support. The work shipped through @qvac/llm-llamacpp@0.24.0 and was included in @qvac/sdk@0.13.0.
