CUDA
Short answer
CUDA is NVIDIA's compute platform. It is usually the fastest way to run a model on an NVIDIA GPU and works on nothing else.
CUDA builds are faster at the large end — on a 7B model, about 16% faster at generation than Vulkan in Nightjar's own measurements — because the kernels are tuned by the company that made the silicon.
The drawbacks are the runtime size, roughly 611 MB against Vulkan's 32 MB, and that an AMD or Intel GPU gets nothing from it at all. A browser cannot assume an NVIDIA card is present, so Vulkan is the default.
Related terms
Other ways people ask this
- what is cuda