[ExI] Watch "GPT 4 is Smarter than You Think: Introducing SmartGPT" on YouTube

[Gadersd]

I wonder if the limitations of GPT4’s accuracy is mainly due to the way it was trained, the number of parameters, or the architecture.

The training is definitely suboptimal as mostly equal weight is put on all the words during training. The human brain focuses its training on the answers, allowing for a tradeoff between the accuracy of answer prediction vs background noise prediction. In some sense GPT focuses just as much on the background noise of words surrounding the answers as the answers themselves.

GPT4 has much fewer parameters than the human brain. It remains to be seen if just additional scaling with endow it with superhuman level reasoning, but I doubt it since it clearly has enough parameters to in a loose way memorize most of the text on the internet.

GPT4’s architecture is the best we have so far as it utilizes parallel hardware the most effectively. I am skeptical that there exists an architecture suitable for our computers that yields a substantial leap over the current state of the art.

Given that GPT4 has essentially memorized well the internet text corpus I suspect that the main obstacle holding it back is its reasoning ability. I think training improvements will provide the most substantial increase in its reasoning ability.

> On May 8, 2023, at 6:22 AM, Jason Resch via extropy-chat <extropy-chat at lists.extropy.org> wrote:
> 
> https://youtu.be/wVzuvf9D9BU <https://youtu.be/wVzuvf9D9BU> 
> 
> Quite interesting, when reflection and step by step processing is used, it's accuracy jumps to exceed the top percentile of human experts in their area of expertise, but GPT-4 achieves this performance in all topics.
> 
> Jason 
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230508/728f9e58/attachment.htm>