[ExI] Language Models on Consumer Hardware

[spike at rainier66.com]

…> On Behalf Of Gadersd via extropy-chat
Subject: [ExI] Language Models on Consumer Hardware

 

>…I am glad to say that it seems that I was wrong when I argued that it would be a while until we could run good large language models on our puny hardware. A Bulgarian programmer managed to hack Meta’s torrented language model LLaMA to reduce the memory footprint to 4 bits per parameter and now the 13 billion parameter version can be run on consumer hardware. See Large language models are having their Stable Diffusion moment (simonwillison.net) <https://simonwillison.net/2023/Mar/11/llama/>  for the full story…

 

 

OK cool, so let us think for a minute about Deep Blue, the chess software that surpassed the best humans in 1997 using a supercomputer.  IMB took it out of service quickly, because we could see that within a coupla years, good powerful workstation class desktops were producing similar results and within a decade processors in cell phones were competing at grandmaster level.

 

Money tournaments with humans, specifically excluding computers, continued after that, but we all knew it was only a matter of time before someone figured out a way to hide a processor and communicate with it somehow in such a way that it was not detectable.  About a decade ago a guy put a device in his shoe, but he was caught.  He refused to remove his shoe and left the tournament, which we interpreted as an admission of guilt.  We suspected someone would eventually put one up his rear or insert it somehow in her female parts, and figure out some tricky I/O, which someone did this past year (the mind boggles.)

 

OK, so… we saw ChatGPT running on massive hardware, but naturally one who has lived thru the experience described above wants to find a way to… well not necessarily cram an AI up my ass but somehow wear a ChatGPT device on my person where it is unobtrusive, and know everything about everything.  Gadersd, can you work out a way to carry whatever is necessary?  Is a cell phone sufficient ya suppose?

 

spike

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230312/e18faf13/attachment.htm>