<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><blockquote type="cite" class="">Given that 200 billion words, perhaps we could make a pastor-bot, and if so, I could dispense religion in arbitrary quantities and make a buttload.</blockquote><div class=""><br class=""></div>Spike, take a look at the image I attached. It is from Deepmind and shows the optimal model size for the amount of available training data in tokens. It seems that your pastor bot if you actually acquired 200B holy tokens would need to be under 10B in size. That would probably yield an interesting model but the capabilities of it would pale in comparison to what we have become accustomed to. GPT4 has about 1T parameters and GPT3 has about 175B parameters. With that said, perhaps 10B is enough for a reasonably capable priest. Don’t expect very good reasoning ability, though I am not sure how common that skill is among priests.<div class=""><br class=""></div><div class=""><img apple-inline="yes" id="96C1846F-3588-4311-B679-901D1A375AC1" src="cid:62F9F0CC-B24F-4144-A835-8B61455D39F3" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 6, 2023, at 12:57 AM, spike jones via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org" class="">extropy-chat@lists.extropy.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><br class=""><br class="">-----Original Message-----<br class="">From: extropy-chat <<a href="mailto:extropy-chat-bounces@lists.extropy.org" class="">extropy-chat-bounces@lists.extropy.org</a>> On Behalf Of Gadersd via extropy-chat<br class="">...<br class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">... Reason I asked: if someone were to create a Bible-bot with input consisting only of a bible and a few dozen commentaries on that collection of books, would that generate actual text sounding like the bible? If so, could we generate something like the Book of Mormon? If so, could not we make a cubic buttload of money off of that?<br class=""></blockquote></blockquote><br class=""><blockquote type="cite" class="">...That could be done but would require an astronomical amount of Bible themed training text... These models need billion or even trillions of tokens/words....<br class=""></blockquote><br class=""><br class="">Gadersd, picture in your mind a room at part of a library, perhaps on the main floor. This room is 88 paces in length, 60 meters and perhaps 10 meters wide with one wall facing in made of glass, so one can see out into the library as well as anyone walking into the library can see into that room we used to call the fish bowl. It wasn't the place for privacy.<br class=""><br class="">In that room, visualize four long narrow tables, like the one in the famous painting:<br class=""><br class=""><a href="https://en.wikipedia.org/wiki/Last_Supper" class="">https://en.wikipedia.org/wiki/Last_Supper</a><br class=""><br class="">(JC: Manservant, table for twenty six. Waiter: But thou numberest only thirteen? JC: Verily, but we all wish to sit on the same side.)<br class=""><br class="">The wall opposite the glass wall is continuous bookshelves, floor to about the height a tall man can easily reach any volume on the top, so perhaps two meters high by about 60 meters. I once estimated the fish bowl contained somewhere around thirty thousand volumes, and all that material was religion stuff, almost all of it specifically writings of Christian writers, much of it bible commentaries written by various brands of religioners. <br class=""><br class="">OK thirty thousand volumes, estimate average of about 600 pages of text, perhaps a thousand words per page, I am getting 2E11 words, or 200 billion words, written by people with a vaguely similar world view.<br class=""><br class="">Here's the reason that collection in the fishbowl interests me: I can probably get most of that stuff on CDs for a reasonable cost. Many religious denominations offer their own teachings and their own spin on religion in that form at a most reasonable price, and much of it might even be public domain, since a lot of it was written a long time ago and has little remaining commercial value or active copyright. Given that 200 billion words, perhaps we could make a pastor-bot, and if so, I could dispense religion in arbitrary quantities and make a buttload.<br class=""><br class="">Gadersd, think that might work? <br class=""><br class="">Two parts to this inquiry: the first one is kinda easy. Is that approach above likely to create a viable religion-bot?<br class=""><br class="">The second part is trickier, but I want to invite commentary on the first part first. Gadersd? What thinkest thou?<br class=""><br class="">spike<br class=""><br class=""><br class=""><br class=""><br class="">_______________________________________________<br class="">extropy-chat mailing list<br class="">extropy-chat@lists.extropy.org<br class="">http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat<br class=""></div></div></blockquote></div><br class=""></div></body></html>