[ExI] ChatGPT 'Not Interesting' for creative works

[spike at rainier66.com]

From: extropy-chat <extropy-chat-bounces at lists.extropy.org> On Behalf Of Gadersd via extropy-chat

Subject: Re: [ExI] ChatGPT 'Not Interesting' for creative works

 

The year-old prediction is useless of course, but the idea is to compensate for the limited calculation ability and bandwidth by giving it more time.

 

>…The analogy does not extend to language models. You cannot compensate for a small model with more computing time. These models have a fixed computing burden that is inversely proportional to model size…

 

OK but the explanation you gave doesn’t support that contention.  Read on please:

 

>…I think you have the wrong intuition … These models are essentially matrix multiplication. Small matrices multiply faster than large matrices…

 

Sure, but my contention is that the distributed model would still multiply the same size matrix.  If we need to multiply a 50x50, that task can be effectively distributed into background computing, but it would take reliable bandwidth and probably a lot of redundancy.  

 

Consider the task of finding the determinant of a 50x50.  That can be distributed among 50 computers each finding the determinant of a 49x49, each of which can be distributed into 49 processors and so on.  Matrix multiplies and inversions can likewise be distributed, but of course it would be a brittle process: any one processor could mess it up.

 

OK idea: get a bunch of investors together who can kick in a few tens of thousands, rent some unused office or warehouse space somewhere, set up a closed system server farm training toward a particular bias agreed upon by the investors.  You would form a scaled down (but still big) GPT which is intentionally trained in material friendly to libertarianism for instance, or believing that causing the extinction of mosquitoes is good but in general causing extinction is bad.

 

Contention: whatever the ChatGPT investors did, a smaller group with less money can do likewise.

 

Given that, one could create a commercial chatbot specialized in training students for instance, or spreading religion, or selling products.  Oh I see mega profits trying to be made here.

 

spike

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230307/d0236958/attachment.htm>