[LINK] How Obama Will Use Web Technology

stephen at melbpc.org.au stephen at melbpc.org.au
Mon Jan 26 22:57:28 AEDT 2009

Re: <http://www.kottke.org/09/01/the-countrys-new-robotstxt-file>

Stil writes,

> The only one which is a possible question-mark is the set of URLs  
> ending in "text". I'm guessing .. Nothing here indicates a change
> in the executive branch's mindset,  just a routine change in the
> structure of the website. And if this came from kottke.org then
> Kottke ought to know better. Stil

Perhaps, Stil .. together Miller & Kottke seem persuasive .. go Obama :)

> White House Unblocks Google; Removes 2400 robots.txt lines from website 
By Jason Lee Miller - Sat, 01/24/2009 - 6:58pm. 


If the outgoing Bush Administration was thought to run a secretive, 
bubble-icious type of White House, the Obama Administration so far is 
proving to be the opposite. 

The www.Whitehouse.gov redesign for greater transparency has already been 
widely noted—Presidential blog and all—but the website is now much more 
open to a new kind of visitor: the search engine spider. 

On Monday, Whitehouse.gov was still blocking search engine access to a 
remendous amount of website information. In all, the robots.txt file used 
the “Disallow” command 2,400 times, blocking search engine access to 
information on earmarks, African American history, photo essays from 
various places and events, first lady initiatives, the budget, defense, 
on and on.

Obviously, if posted on the White House website, none of this information 
would be considered classified, or even sensitive, so it’s unclear why 
Bush’s web crew felt the need to prevent the site from being searchable. 

Regardless, all search crawler barriers were removed with the Bushes’ 
furniture, the “Disallow” command lines reduced from 2,400 to basically 

Requests for comment and/or explanations from prior and current 
administrations were not returned. Meanwhile, it appears President Obama 
will be able to keep his Blackberry after all—with some super-encryption 
functionality added to it.  

Cheers, people
Stephen Loosley
Victoria, Australia

More information about the Link mailing list