<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Correction, I meant to say that the human brain has 100 trillion parameters, not 1 trillion.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar 6, 2023, at 11:24 AM, Gadersd <<a href="mailto:gadersd@gmail.com" class="">gadersd@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">The human brain has roughly 1 trillion parameters so large language models are still a few orders of magnitude short of the human brain. It should be noted however that not all human brain connections perform language tasks, so achieving a fully human level of natural language understanding should require fewer than 1 trillion parameters.<div class=""><br class=""></div><div class="">Toy models can and have been trained in parallel across consumer computers, but I think you would be disappointed in their intelligence as compared to ChatGPT.</div><div class=""><br class=""></div><div class="">For example I tried a 6 billion parameter model GPT-JT, accessible at <a href="https://huggingface.co/spaces/togethercomputer/GPT-JT" class="">https://huggingface.co/spaces/togethercomputer/GPT-JT</a>.</div><div class="">Prompt: "s<b class="">olve 2x+3=-1 step by step. 2x="</b></div><div class=""><b class="">Answer: "</b><i class="">1, so x=1/2.<br class=""><br class="">A:<br class=""><br class="">The answer is $1”</i></div><div class=""><i class=""><br class=""></i></div><div class="">This model was trained in parallel as you have suggested. Not very useful, is it?<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Mar 5, 2023, at 10:17 PM, spike jones via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org" class="">extropy-chat@lists.extropy.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><blockquote type="cite" class="">... Gadersd via extropy-chat<br class=""></blockquote>...<br class="">Subject: Re: [ExI] ChatGPT 'Not Interesting' for creative works<br class=""><br class=""><blockquote type="cite" class="">...Computing technology is not advanced enough for consumer computers to run the powerful models. Consumer computers do not have the bandwidth and GPU FLOPS to run the good models. It isn’t a matter of speed, consumer computers just cannot run the big models. The best you could do is run a toy model with maybe a billion parameters. Such toy models are completely dumb compared to ChatGPT and can barely string coherent sentences together...<br class=""></blockquote><br class=""><br class=""><br class="">OK so what if... we get a number of us running in parallel. A toy version with a billion parameters, well OK then, a billion is about three orders of magnitude more parameters than my beleaguered meat brain has (as far as I know (hell I don't even know what my own parameters are)) and yet it seems to somehow write fun stuff on occasion.<br class=""><br class="">spike<br class=""><br class=""><br class="">_______________________________________________<br class="">extropy-chat mailing list<br class=""><a href="mailto:extropy-chat@lists.extropy.org" class="">extropy-chat@lists.extropy.org</a><br class=""><a href="http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat" class="">http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat</a><br class=""></div></div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></body></html>