[extropy-chat] how does google find out everything?

[Eugen Leitl]

On Fri, Aug 18, 2006 at 07:10:57PM +0100, Russell Wallace wrote:

>    from the beginning. (I vaguely recall the cycle time for the major
>    search engines was measured in months, back in the early days; I'm
>    guessing Google's available bandwidth has outpaced the growth of the
>    web so the cycle time would now be shorter.)

I typically notice that web spiders hitting by a spike on the
switch port (and soon, by a traffic quota warning). The major
engines (Yahoo, Microsoft, Google) typically hit 1-2 times/month.
I have a lot of data on my site, so about 90% of my traffic
is due to spiders.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820            http://www.ativel.com
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 191 bytes
Desc: Digital signature
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20060822/a64a54f8/attachment.bin>