[ExI] Fwd: GPT-4 gets a B on Scott Aaronson's quantum computing final exam

Giovanni Santostasi gsantostasi at gmail.com
Sat Apr 29 00:33:20 UTC 2023


As I said I was blown away when GPT-4 told me it made a mistake by coloring
below the horizon in his helicopter drawing with the same color of the sky.
I simply told it that I think it made a mistake and to tell me what I
thought it was. Not sure how it went through the steps of analyzing this,
in particular, because it was all SVG code. This is not something that it
could have been solved by accessing some archived information. Gordon never
addressed how an autocomplete can achieve this type of analysis and
recognize a subtle error like this. As I mentioned before if were sitting
with a child and he went through this reasoning process we would recognize
the child as highly intelligent (of course it depends on the age, but I
would say even a 4-5-year-old child would have problems in coloring
different parts of the drawing consistently).

On Fri, Apr 28, 2023 at 5:16 PM Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

>
>
> On Fri, Apr 28, 2023, 5:53 PM efc--- via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
>>
>>
>> On Thu, 27 Apr 2023, Jason Resch via extropy-chat wrote:
>>
>> > I thought this was interesting and relevant to discussions of what
>> GPT-4 understands.
>> > Here a professor graded it's responses to the final exam questions of a
>> test which was not in the training set used by GPT since it
>> > was never put online.
>> >
>>
>> But were any papers or data related to the subject in the training set?
>>
>> I did this myself for my own exams and alpaca passed it. My exam was not
>> "in it" but since knowledge of the domain (linux) is in it, of course I
>> can answer the questions.
>>
>
> I'm sure some related material is out there, but I've also seen GPT solve
> logic puzzles written from scratch for the purposes of testing GPT, so it
> must have some reasoning ability beyond rote memorization.
>
> Jason
>
>> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230428/14a547fb/attachment.htm>


More information about the extropy-chat mailing list