[ExI] Language Models on Consumer Hardware

[Gadersd]

I am glad to say that it seems that I was wrong when I argued that it would be a while until we could run good large language models on our puny hardware. A Bulgarian programmer managed to hack Meta’s torrented language model LLaMA to reduce the memory footprint to 4 bits per parameter and now the 13 billion parameter version can be run on consumer hardware. See Large language models are having their Stable Diffusion moment (simonwillison.net) <https://simonwillison.net/2023/Mar/11/llama/> for the full story.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230312/a57664a9/attachment.htm>