[extropy-chat] how does google find out everything?

[Russell Wallace]

On 8/18/06, spike <spike66 at comcast.net> wrote:
>
> Internet gurus, how does Google get its info?  If anyone puts up any
> website, Google knows somehow, right?  It doesn't take any action on the
> part of the author.  Google knows of every post on ExI as well as any
> other
> chat group too?  It knows when you are sleeping, it knows when you're
> awake.
>
> How does it do that?


By spidering. A program (called a spider) starts off with a list of known
web pages, then checks all the pages they link to, then all the pages those
link to, etc; then when that process is complete, restarts from the
beginning. (I vaguely recall the cycle time for the major search engines was
measured in months, back in the early days; I'm guessing Google's available
bandwidth has outpaced the growth of the web so the cycle time would now be
shorter.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20060818/713c8c69/attachment.html>