<div dir="ltr"><div><div><div>Reinforcement learning has proven very powerful in AI. I have no reason to believe that they have not accomplished what they claimed. These results flow naturally from where the field has been headed, and is not the leap of faith it first appears to be. While I am excited to hear the results, and am impressed, it is still very brittle compared to generalized intelligence IMO. This type of program still needs to be trained on a very specific problem, and while it is able to be generalized as a technique, there is no thought process going on behind it.<br></div><div><br></div><div>The program is basically using a combination of deep learning neural nets and survival of the fitness to replace the best player during simulation training with any new one that beats the old one by greater than a certain cutoff (I believe it was 55% in the paper). You have a massive number of iterations of simulated gameplay that are minimizing a loss function via gradient descent (which in a perfect world finds a global minima) that is keeping the training of the better player in each round.<br><br></div>There have been many remarkable things accomplished with deep/reinformcement learning. It's quite startling at first glance to think that an end goal of minimizing a loss function can generate so much razzle dazzle, but the math behind these systems is actually not that complex. It is essentially matrix multiplication combined with a nonlinear activation function on the forward pass through a neural network, followed by gradient descent using calculus to backfeed new weights throughout the network, and then having the machine play many, many matches against itself, rinse and repeat.<br><br></div>John-<br><br></div>You may find the ideas at this link interesting based on your last sentence:<br><a href="https://medium.com/@karpathy/software-2-0-a64152b37c35">https://medium.com/@karpathy/software-2-0-a64152b37c35</a><br><div><div><br><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 7, 2017 at 10:42 AM, spike <span dir="ltr"><<a href="mailto:spike66@att.net" target="_blank">spike66@att.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div link="blue" vlink="purple" lang="EN-US"><div class="m_9168227948497600670WordSection1"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal"><b>From:</b> extropy-chat [mailto:<a href="mailto:extropy-chat-bounces@lists.extropy.org" target="_blank">extropy-chat-bounces@<wbr>lists.extropy.org</a>] <b>On Behalf Of </b>John Clark<br><b>Sent:</b> Thursday, December 07, 2017 7:16 AM<br><b>To:</b> ExI chat list <<a href="mailto:extropy-chat@lists.extropy.org" target="_blank">extropy-chat@lists.extropy.<wbr>org</a>><br><b>Subject:</b> Re: [ExI] alpha zero<u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><div><div><p class="MsoNormal"><span style="font-family:"Arial",sans-serif"><u></u> <u></u></span></p></div><div><div><span class=""><p class="MsoNormal">On Wed, Dec 6, 2017 at 9:10 PM, spike <<a href="mailto:spike66@att.net" target="_blank">spike66@att.net</a>> wrote:<u></u><u></u></p><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><div><div><p class="MsoNormal"><span style="font-family:"Arial",sans-serif">> <u></u><u></u></span></p></div><p class="MsoNormal">DeepMind, the same outfit which made the learning Go program is now claiming they did the same trick with chess. I don’t know if I believe it (rather I vaguely do not believe it) but it is being reported on a very reliable chess site:<u></u><u></u></p><p class="m_9168227948497600670gmail-msonormal"><a href="https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess" target="_blank">https://en.chessbase.com/post/<wbr>the-future-is-here-alphazero-<wbr>learns-chess</a><u></u><u></u></p><p class="m_9168227948497600670gmail-msonormal">They are claiming that it learned from only the rules of chess in 24 hours. I just don’t see how it could have mastered the collective human experience over more than 500 years in 24 hours.<u></u><u></u></p><p class="m_9168227948497600670gmail-msonormal">If Deep Mind really did this, it’s the most impressive computer learning feat I have ever seen.<u></u><u></u></p></div></div></blockquote><p class="MsoNormal"><span style="font-size:13.5pt">>…You're right Spike </span><span style="font-size:13.5pt;font-family:"Arial",sans-serif">it's</span><span style="font-size:13.5pt"> simply amazing! <u></u><u></u></span></p><p class="MsoNormal"><u></u> <u></u></p></span><p class="MsoNormal">I still haven’t convinced myself it is true. I think highly of the source that reported it, but they can be fooled. They played Stockfish, which is a very highly respected program with a lotta lotta programmed-in chess wisdom. To figure out all that in a day requires some powerful inference activity. John I am putting myself in the camp of hope it’s true, but estimate 70% chance it isn’t. I don’t know how the hell they did this if true.<u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">><span style="font-size:13.5pt">…</span><span style="font-size:13.5pt;font-family:"Arial",sans-serif"> And if you ever hear that it's starting to treat optimizing computer code as a game then you may be hearing the opening notes of the Singularity. This is big…John K Clark<u></u><u></u></span></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Sure and is there any reason why we shouldn’t treat code optimization as a game? It is a clearly-definable goal: we can set the task to give a known outcome, give it a time to beat and a memory allocation to beat, may the best machine win. It’s one of those new sports I have been yakking about for years, a great example of geek Olympics. <u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">I want robot gymnastics too. Whooda thunk that would just appear like it has?<u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal"><a href="http://www.cnn.com/videos/cnnmoney/2017/11/17/atlas-boston-dynamics-robot-backflip-cnntech.cnnmoney" target="_blank">http://www.cnn.com/videos/<wbr>cnnmoney/2017/11/17/atlas-<wbr>boston-dynamics-robot-<wbr>backflip-cnntech.cnnmoney</a><u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">We could have a code-athlon, where the game is to write the best and most efficient code, then let computers play against each other and against humans.<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></p><span class="HOEnZb"><font color="#888888"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">spike<u></u><u></u></p></font></span></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p><div><div><p class="MsoNormal"><span style="font-size:18.0pt;font-family:"Arial",sans-serif"><u></u> <u></u></span></p></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div><br>______________________________<wbr>_________________<br>
extropy-chat mailing list<br>
<a href="mailto:extropy-chat@lists.extropy.org">extropy-chat@lists.extropy.org</a><br>
<a href="http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat" rel="noreferrer" target="_blank">http://lists.extropy.org/<wbr>mailman/listinfo.cgi/extropy-<wbr>chat</a><br>
<br></blockquote></div><br></div>