<div dir="ltr"><div>Samantha,</div><div><br></div>I suppose it depends on what you mean by "implement". <div><br></div><div>Consider the difference between </div><div><br></div><div>F(n)=1√5[(1+√52)n−(1−√52)n]<br></div><div><br></div><div>And </div><div><br></div><div>def F(n)<br> if n < 2<br> return n<br> else<br> return F(n - 1) + F(n - 2)<br> end<br>end<br></div><div><br></div><div>Both implement the exact same function - finding the nth Fibonnaci number, but using radically different code. The first is implementation of the second. It happens to be a lossless compression of the second. But what if it was an imperfect implementation, delivering occasionally different results from the second. I think it would still be fair to call it an implementation, albeit an imperfect one.</div><div><br></div><div>Similarly, the F(tokenstream -> tokenstream) learned by a LLM is an implementation, albeit imperfect, of the system that generated its training data. And if that underlying system (human linguistic computation) is Turing complete (modulo memory size), then thel earned function may be as well.</div><div><br></div><div>[If you want to reserve the definition of "implementation" for functions that are precisely correct over their entire range, I won't argue, but I think "implementation" is a useful concept if it can encompass some fuzziness.]]</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 20, 2023 at 2:17 AM sjatkins via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org">extropy-chat@lists.extropy.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-family:Arial,sans-serif;font-size:14px"><br></div>
<div style="font-family:Arial,sans-serif;font-size:14px">
<div></div>
<div>
</div>
</div>
<div style="font-family:Arial,sans-serif;font-size:14px"><br></div><div>
------- Original Message -------<br>
On Wednesday, April 19th, 2023 at 3:50 PM, Darin Sunley via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org" target="_blank">extropy-chat@lists.extropy.org</a>> wrote:<br><br>
<blockquote type="cite">
<div dir="ltr"><br></div><div dir="ltr"><div><br></div><br><div><br></div><div>That seems like it's the secret sauce - the ginormous hint that got ignored by data/statisticis-centric ML researchers. When you learn how to map token streams with significant internal structure, the function your neural net is being trained to approximate will inevitably come to implement at least some of the processing that generated your token streams. <br></div></div></blockquote><div style="font-family:Arial,sans-serif;font-size:14px;color:rgb(0,0,0)"><br></div><div style="font-family:Arial,sans-serif;font-size:14px;color:rgb(0,0,0)">I think this is too strong a claim. The LLM system will simulate similar features as those found in what generated the token systems but I would argue this in not an implementation of those features. If you will it is more like a ghost image or echo of those features. The mapping is very much not the territory. <br></div><div style="font-family:Arial,sans-serif;font-size:14px;color:rgb(0,0,0)"><br></div><div style="font-family:Arial,sans-serif;font-size:14px;color:rgb(0,0,0)">- samantha<br></div><div><br></div><div><br></div><blockquote type="cite">
</blockquote><br>
</div>_______________________________________________<br>
extropy-chat mailing list<br>
<a href="mailto:extropy-chat@lists.extropy.org" target="_blank">extropy-chat@lists.extropy.org</a><br>
<a href="http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat" rel="noreferrer" target="_blank">http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat</a><br>
</blockquote></div>