[LINK] Latency in indexing robots

Eric Scheid eric.scheid at ironclad.net.au
Wed Dec 6 15:00:31 AEDT 2006

On 6/12/06 1:49 PM, "Antony Barry" <tony at tony-barry.emu.id.au> wrote:

> Does anybody have any idea of the sort of delay before the robots
> will come back and try the pages again??

have you tried telling the bots to come looky?


also, consider using google's sitemap thingy


   "Sitemaps are an easy way for webmasters to inform search engines
    about pages on their sites that are available for crawling. In its
    simplest form, a Sitemap is an XML file that lists URLs for a site
    along with additional metadata about each URL (when it was last
    updated, how often it usually changes, and how important it is,
    relative to other URLs in the site) so that search engines can more
    intelligently crawl the site."

now supported by Google, Yahoo!, and Microsoft.

no idea if the sitemaps protocol is supported by Pandora yet.


More information about the Link mailing list