Musings on IT, data management, whitewater rafting, and backpacking

Thursday, March 22, 2012

When triple redundancy isn't enough


Last year we implemented a data archiving system, where Availability and Survivability were important design parameters. We had one primary server in California, with mirrors in Florida and Massachusetts – triple redundancy. Life was good for many months.

Right now, all three systems have been down for over 24 hours. 

Here's the story.

Sunday, March 11, 2012

EPO buttons not intuitive

October 17, 1989, 5:04 pm. The Loma Prieta earthquake severely shakes our buildings in the Palo Alto foothills, and cuts off electric power in the area.

Soon after the shaking stops, I go to the pitch-black, silent server room with a flashlight, and push the Big Red Button, expecting to cut off power and clean up the mess when we return to work.

A few days later I return to a big surprise ...

Monday, March 5, 2012

You don't save money buying computers

In the New York Times today "Digital Records May Not Cut Health Costs".

I've watched or run many projects that were justified with: "If we computerize this process, we'll save lots of money".

Never worked. Ever. In 40 years.

If you are lucky, you might do things faster, or better, but you won't save money. Usually, the project fails to do anything faster or better, and it costs a lot more than the old way.

Ironic that the rest of the world keeps re-learning this lesson.