[LINK] Navitaire was a Virgin, and now Blue as well

[Peter Bowditch]

Another true story.

An extremely large database is delivered from Melbourne to Sydney on CD, 
installed on the server and about a month spent by several people cleaning 
it up.

I get a panic call from the Sales Manager on a Monday: The IT people 
upgraded the server over the weekend. For some reason they dug out the CD 
from Melbourne and reinstalled the database. A month's work has been lost. 
Help.

I say: Find the backup tape from Friday night and lock it in a drawer so 
that nobody accidentally overwrites it. Then get the IT people to do a 
restore.

A little later, the IT man is on the phone: How do we do a restore from 
tape?

After I recover my breath I say: For this particular piece of software the 
backup runs on the server but the restore has to be done from a 
workstation. If you can't locate the workstation with the restore software 
put it on another one and get restoring.

A little later the IT man is back on the phone: I've installed the restore 
software, now how do I do a restore?

After I get my breath back again I say: It's like Windows Explorer, you 
locate the files on the tape in one frame and copy them to the server in 
another.

Later still: I can't seem to get it to work. Can you come over and fix it?

I drive from Parramatta to Alexandria, sit in front of a workstation and 
note that there is NO BACKUP of the database on the tape. I few enquiries 
elicit the information that the backup software was set to only try once 
if a file was open and then skip it if it was.

In this particular place they were so security conscious that everyone had 
a password-protected screen saver than came on after one (1) minute of 
inactivity. Because the staff quite rightly found this to be a PITA, it 
was almost inevitable that someone would go home at night while leaving 
his computer on and the database open.

So these people had not only never tested their backup but hadn't even 
installed the recovery software to try, didn't know how to do a restore 
and had configured the backup to inevitably miss things.

Shall I mention this company's principal business? Why not?

Off-site data and record storage.

By the way, the backup tapes were religiously taken out of the server each 
morning and deposited in a very secure, fire and thief-proof safe in 
another building. It's just a pity that what was on them was useless.

>   True story.
> 
> 1. Moving offices from Level Y to Level X in about 1997.
> 
> 2. Nightly backups were on a tape but nobody had tested the tape in ???
> 
> 3. Server dropped down the stair well.
> 
> 4. Backup would not restore from tape in new office.
> 
> It turned out that someone had been a wee bit too aggressive with the 
> spanner when they put the tape drive in the first rack. That bent the 
> drive's chassis slightly out of shape. The tapes would not align in the
> new location, because the installer had been more kind. It took days for
> a technician to work out what had gone wrong...
> 
> RC
> 
> On 12/10/10 5:59 PM, Stilgherrian wrote:
> > On 12/10/2010, at 10:33 AM, Steven Clark wrote:
> >>> [So, finally, we have the answer.
> >>> [The backup and recovery plan had never been tested.]
> >> which is fail! in both senses>.<  it's not a backup and recovery
> system
> >> if you're not even sure it works. *headdesk*
> >>
> >> though this is not as surprising as it ought to be. so very often,
> >> technologies are assumed to 'just work'.
> > Indeed, Steven. I had wondered, when I saw in Teh Meedjuh and other
> places, so many commentators saying that not testing your backup and
> disaster recovery was "stupid" and worse, how certain they themselves
> were that their own organisations weren't in exactly the same boat.
> >
> > Gentle Linkers, when was the last time you tested the process for
> restoring your own computer from backup, including data, applications and
> their configuration? And how long did it take?
> >
> > [My own answers, for the record, are about 18 months, and most of a
> day, though I am happy that I could access my data more quickly by other
> means if necessary.]
> >
> > Stil
> >
> >
> 
> 
> _______________________________________________
> Link mailing list
> Link at mailman.anu.edu.au
> http://mailman.anu.edu.au/mailman/listinfo/link
> 
> 


-- 
Peter Bowditch
The Millenium Project - http://www.ratbags.com/rsoles
Australian Council Against Health Fraud - http://www.acahf.org.au