[ExI] Language Models on Consumer Hardware

[Gadersd]

Someone managed to run the 13 billion parameter model on a 3.6 GHz processor with 8 GB ram so those phone specs are close! But be aware that GPT3 is 165 billion parameters so we are still a while from running something as big as ChatGPT on a phone. The 13 billion parameter models are approaching usability though.

> On Mar 12, 2023, at 4:20 PM, BillK via extropy-chat <extropy-chat at lists.extropy.org> wrote:
> 
> On Sun, 12 Mar 2023 at 17:30, spike jones via extropy-chat
> <extropy-chat at lists.extropy.org> wrote:
>> 
>> OK, so… we saw ChatGPT running on massive hardware, but naturally one who has lived thru the experience described above wants to find a way to… well not necessarily cram an AI up my ass but somehow wear a ChatGPT device on my person where it is unobtrusive, and know everything about everything.  Gadersd, can you work out a way to carry whatever is necessary?  Is a cell phone sufficient ya suppose?
>> 
>> spike
>> _______________________________________________
> 
> 
> I am not a smartphone expert, but.....
> The Samsung Galaxy S23 Ultra (about 1500 USD) specs say
> CPU: Snapdragon 8 Gen 2 Mobile Platform for Galaxy
> (Up to 3.36 GHz)
> RAM: 8GB / 12GB
> Storage: 256GB / 512GB / 1TB
> 
> and the Samsung Dex system allows connection to monitor, keyboard and mouse.
> 
> So phones are getting pretty powerful nowadays.
> 
> 
> BillK
> 
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat