Our engineering team has been optimising the QVAC audio stack, so we measured it. We put qvac-fabric-speech.cpp against the open-source engines people run for transcription, speech synthesis and music generation, on four machines and six GPU lanes, timing every engine from outside its own process so that nobody's own stopwatch is quoted. QVAC leads on the GPU lanes we measured, across every vendor. Here is each number, and how we took it.
Our engineering team is continuously optimising the QVAC audio stack.
The optimisations live in qvac-fabric-speech.cpp, the open-source engine underneath it, and we benchmarked its three main capabilities: transcription, speech synthesis and music generation.
github.com/tetherto/qvac-fabric-speech.cpp
We compared them on four machines and six GPU lanes, against the open-source engines people run for each job. Here are the results.
We transcribed a 98-second clip with Parakeet TDT 0.6b v3.
On a fanless MacBook Air it takes us 756 ms and audio.cpp 865 ms. That machine has no fan and its results swing 7 to 24 percent between runs, so we read those two as level. transcribe.cpp takes 1,746 ms on the same laptop. parakeet.cpp, the engine LocalAI ships for this model, has no working Metal path at the commit LocalAI pins, and takes 8.2 s.
The gap opens up on the other five GPU lanes, where Fabric leads audio.cpp by 1.85x to 2.81x. It leads transcribe.cpp on all six, by 2.1x to 8.5x. On an RTX 5090 over CUDA the clip takes 58 ms.
Supertonic 3 speaks 26.5 seconds of text in 0.65 s on that MacBook Air, against 4.70 s for audio.cpp.
Fabric finishes synthesis first on Metal, CUDA and Vulkan, 2.4x to 7.2x faster end to end than audio.cpp, and loads its model 2.3x to 4.8x faster.
Generating a song takes us 9.0 s on the MacBook against 10.0 s for acestep.cpp, and 9.7 s against 10.0 s once our model load is counted in.
On a Strix Halo we generate the same song in 2.3 s against 4.8 s for acestep.cpp, and we come out faster on every one of the 60 prompt-and-lane combinations.
On transcription every engine ran at its own shipping defaults with its own weights. On synthesis both engines ran at f16, the only precision their two packages share. We quote no engine's own timer, including our own.
Versions tested: audio.cpp 0.7.3, transcribe.cpp 0.2.3, parakeet.cpp at the commit LocalAI pins, acestep.cpp at upstream HEAD. We forced acestep.cpp's release build, since its own script leaves the build type unset.
Try out QVAC Fabric Speech now: https://github.com/tetherto/qvac-fabric-speech.cpp
Part 4 of Local AI 101: how to put local AI to work in a company, for a solo founder, a team in one office or a remote team, with the machines, the costs for 20 people and a go-live checklist.
Read More
Paolo Ardoino answered the community’s questions about QVAC and local AI: why AI should run on the devices people own, who pays for the compute and what a cheap phone can do today.
Read More
TranscribePsy is a 254M-parameter speech recognition model that turns speech into text on a phone or a laptop, in 25 European languages on a laptop and 8 on a phone.
Read MorePart 4 of Local AI 101: how to put local AI to work in a company, for a solo founder, a team in one office or a remote team, with the machines, the costs for 20 people and a go-live checklist.
Read MorePaolo Ardoino answered the community’s questions about QVAC and local AI: why AI should run on the devices people own, who pays for the compute and what a cheap phone can do today.
Read MoreTranscribePsy is a 254M-parameter speech recognition model that turns speech into text on a phone or a laptop, in 25 European languages on a laptop and 8 on a phone.
Read MoreStay updated
New versions, breaking changes and migration notes - straight to your inbox. No spam, unsubscribe anytime.