Honey, I shrunk the LLM! A beginner's guide to quantization – and testing it
Hands on Just be careful not to shave off too many bits ... These things are known to hallucinate as it is
AI + ML14 Jul 2024 | 5