Blog

News, tutorials & more.

All categories Release Feature Technical

Aug 20, 2026 Technical

Use fewer threads for CPU inference

How many worker threads should you use for CPU inference? Not all of them.

Jun 1, 2026 Technical

LLM, give me a JSON. Make no mistakes.

So how exactly do you make your LLM output a JSON? What happens under the hood? And how do you make it reliable and fast? Diving into constrained sampling.

May 14, 2026 GGUF open-source Technical

What's in a GGUF, besides the weights - and what's still missing?

What extra stuff is needed to properly run a language model? Besides the weights of a language model, what is the gguf metadata that we need to parse and use?