[LINK] Latency in indexing robots

Craig Sanders cas at taz.net.au
Thu Dec 7 08:11:38 AEDT 2006

On Wed, Dec 06, 2006 at 07:53:00PM +1100, Roger Clarke wrote:
> Surely they'd use the HTTP HEAD method to ask for last date of change?

some robots do, some don't.

> In fact, you'd expect them to use UDP, in which case the overheads of
> session creation and tear-down aren't involved, and the packet-sizes
> are very small.

http is a TCP protocol, not UDP.


craig sanders <cas at taz.net.au>           (part time cyborg)

More information about the Link mailing list