Technical
Use fewer threads for CPU inference
How many worker threads should you use for CPU inference? Not all of them.
Technical
Updates and technical notes from the NobodyWho team. Subscribe via RSS.
Technical
LLM, give me a JSON. Make no mistakes.
So how exactly do you make your LLM output a JSON? What happens under the hood? And how do you make it reliable and fast? Diving into constrained sampling.
GGUF / open-source / Technical
What's in a GGUF, besides the weights - and what's still missing?
What extra stuff is needed to properly run a language model? Besides the weights of a language model, what is the gguf metadata that we need to parse and use?