[LINK] Remove Rupert from Results Campaign

Wed Nov 25 18:15:46 AEDT 2009

Tom W mentioned:

> To prevent robots from crawling your site, add the following
>  directive to your robots.txt file:
> 
>  User-agent: *
>  Disallow: /

Earlier this year I lost a web site management deal, and "getting an 
expert" was mentioned in the rationalisation. (I did my first web site in 
about 1994. The person who took over from me is under 20, I'm told.) For 
about twice what I charge for a yearly support contract the clients got a 
Wordpress template (yes, just a template) and then had to reload all the 
site content (several hundred pages) themselves. The last I heard they had 
called in some other experts so that more money could be spent replicating 
something that worked perfectly well in the site that I had managed (and 
for which all HTML, JavaScript and Perl had been handed over).

Now, to get to the point.

Would you like to guess at the content of the robots.txt file on the new 
server?

That's right:

User-agent: *
Disallow: /

And it gets even better. The Wordpress template used to produce every page 
on the site contains a '<meta name="robots" value="noindex">' line in the 
<head> section.

Did I mention that when I ran the site it was indexed weekly by Google 
because my CMS maintained an up-to-date sitemap.xml file?

I thought about notifying the ex-client of the problem, but then I 
remembered that I had been replaced by an expert. Professionalism is a 
nice feeling, but sometimes Schadenfreude is even better.

-- 
Peter Bowditch
The Millenium Project - http://www.ratbags.com/rsoles
Australian Council Against Health Fraud - http://www.acahf.org.au