[LINK] Remove Rupert from Results Campaign
Peter Bowditch
peter at ratbags.com
Wed Nov 25 18:15:46 AEDT 2009
Tom W mentioned:
> To prevent robots from crawling your site, add the following
> directive to your robots.txt file:
>
> User-agent: *
> Disallow: /
Earlier this year I lost a web site management deal, and "getting an
expert" was mentioned in the rationalisation. (I did my first web site in
about 1994. The person who took over from me is under 20, I'm told.) For
about twice what I charge for a yearly support contract the clients got a
Wordpress template (yes, just a template) and then had to reload all the
site content (several hundred pages) themselves. The last I heard they had
called in some other experts so that more money could be spent replicating
something that worked perfectly well in the site that I had managed (and
for which all HTML, JavaScript and Perl had been handed over).
Now, to get to the point.
Would you like to guess at the content of the robots.txt file on the new
server?
That's right:
User-agent: *
Disallow: /
And it gets even better. The Wordpress template used to produce every page
on the site contains a '<meta name="robots" value="noindex">' line in the
<head> section.
Did I mention that when I ran the site it was indexed weekly by Google
because my CMS maintained an up-to-date sitemap.xml file?
I thought about notifying the ex-client of the problem, but then I
remembered that I had been replaced by an expert. Professionalism is a
nice feeling, but sometimes Schadenfreude is even better.
--
Peter Bowditch
The Millenium Project - http://www.ratbags.com/rsoles
Australian Council Against Health Fraud - http://www.acahf.org.au
More information about the Link
mailing list