[ExI] Training GPT-3 to play nice - it's difficult!

[spike at rainier66.com]

-----Original Message-----
From: extropy-chat <extropy-chat-bounces at lists.extropy.org> On Behalf Of
BillK via extropy-chat
Sent: Monday, November 2, 2020 9:27 AM
To: ExI chat list <extropy-chat at lists.extropy.org>
Cc: BillK <pharos at gmail.com>
Subject: Re: [ExI] Training GPT-3 to play nice - it's difficult!

On Mon, 2 Nov 2020 at 17:12, spike jones via extropy-chat
<extropy-chat at lists.extropy.org> wrote:
>
>
> BillK, do you know what is the best platform that we can get to?  Some
kind of public domain something where we give it the feedstock text?
>
> spike
> _______________________________________________


>...Training GPT-3 is not easy. You need billions of text sentences. Like
the whole of reddit.......

>...Smaller groups of text would quickly lead to repetition of similar
phrases. Like the early chatbots.

>...BillK

_______________________________________________


OK, so a sentence is about... say... 20 words?  And a word is about or 6
characters and a character is 8 bits so where are we, about a kilobit per
sentence?  So we need billions of sentences, so we need archives of a
trillion bits or a terabit or 100 gigabytes of text?  I think we can do
this.  We can mix ExI with Mensa and such as that, lotsa the geek sites.
How big is the ExI archive? 

It occurred to me that we would need to mix a lotta other stuff in there, or
it would get caught right away.  It would use some of my quirky text
patterns such as using uncommon or archaic terms: constable, harlot, sorta
kinda, bride, larvae, prole, that kinda thing.  It would be so busted, right
away, if we can't figure out a way to get that outta the feedstock text.
It's too late for me to stop using quirky signature writing patterns,
tragically too late.  It's in there now.  All my writings are of no value
for doing Turing test training, damn.  

{8-[

spike