Using Hugging Face models with Nightjar
Short answer
Download a GGUF file from Hugging Face and add it from disk. Repositories that publish only safetensors need converting first — Nightjar loads GGUF.
What to download
Look for a GGUF repository, usually named with a GGUF suffix, and pick the quantisation that fits your memory: file size plus about 1.1 GB has to be under your VRAM.
If only safetensors are published, that is the training format rather than the local-inference one, and it needs converting before any llama.cpp tool will load it.
Adding it
Add the model from disk in the picker. Metadata is read from the file, so context length and sizing work the same as for bundled models.
Worth reading before you download
The model card: what it was trained on, which languages it supports, whether it reads images, and the licence. A card listing no weaknesses is marketing.
Vision support in particular is a property of the model. A text-only model will not read images however you import it.
Other ways people ask this
- hugging face browser
- use hugging face models in browser
- I already have models downloaded in Hugging Face. Can a browser use them directly?