Shirt Pocket Discussions

Shirt Pocket Discussions (https://www.shirt-pocket.com/forums/index.php)
-   General (https://www.shirt-pocket.com/forums/forumdisplay.php?f=6)
-   -   A SuperDuper Back Up Strategy (https://www.shirt-pocket.com/forums/showthread.php?t=1356)

Bagelturf 06-11-2006 01:53 PM

A SuperDuper Back Up Strategy
 
Here is the full text of my latest blog entry: a strategy for using SuperDuper to make back ups:

Having given up on network back ups some time ago, I have been experimenting with and formulating back up strategies for the various machines around here. The goal is to back up quickly and be able to recover quickly without spending too much money and without being forced into software upgrades due to OS changes (ahem, Retrospect, I'm talking to you here).

Here is what I have come up with. Note that it is for back up only, not archiving. In other words, the immediate goal is to recover from failure, not store data long-term.

Each machine has its hard drive partitioned into a boot volume and a Scratch volume. The Scratch volume is never backed up and is typically 20% of the drive. It's a place for big files, anything that is generated by software from backed up files, temporary items, DV captures, etc. This strategy prevents unnecessary backing up of large temporary data.

Each machine gets a Firewire drive that is at least 25% bigger than the amount of data on the boot volume. The Firewire drive has a single partition. The drive is left turned on all the time, but the volume is unmounted. The machine is set to wake up at 5am and SuperDuper is set to run a scheduled copy at 5:31am using a smart copy. What SuperDuper does is to mount the drive, compare the firewire contents with the boot drive content, copy and delete as appropriate, then unmount the drive. Typically this takes fifteen to thirty minutes once the initial copy of the whole drive is performed.

What this buys me is near-instant recovery from a hard drive failure. Recovery works like this: HD fails. Boot machine with Option held down and select the Firewire drive. Machine boots from Firewire drive. I know exactly up to when good data is available, and therefore what I have to do to fill in the missing pieces. I lose a maximum of one day's work. And I can lose less if I schedule more copies during the day, such as at lunch time. It is not that expensive because the Firewire drives can be small and old.

What this does not do is handle catastrophic failures such as as fires, so there is an extra step. For off-site back ups I could maintain an extra set of Firewire drives and switch them with the on-site ones periodically. I actually don't do this because a) it means a lot of drives lugged about, and b) I don't have that many old drives. So instead I have two large capacity new drives (750G each) and have at most one of them on-site at any time.

On those drives I store sparse disk images, one per boot volume I need to back up. Sparse disk images grow as data is added to them, but do not shrink as it is deleted (but a Terminal command line can be used to shrink them back). Periodically I back up the machines, again using SuperDuper's smart copy, this time copying the hard drive to the mounted sparse disk image. In addition, to being convenient and not wasting space, sparse disk images give me the ability to encrypt the data. So all of my disk images are encrypted, ensuring that a lost or stolen back-up drive is no greater a loss than the value of the hardware. The large capacity drives get swapped around each time all the machines have been backed up.

Another convenience of this strategy is that the downtime on the machines is very small. The back up to the sparse disk images on the large capacity drive need not use any time at all on the machine it is backing up, since the Firewire drive that is sitting next to it can be disconnected (it is not even mounted most of the time), carried to another machine, the back up done there, and the drive returned.

Recovery from sparse disk images is easy too, but more time-consuming. Assuming that the machine and its local Firewire drive have been destroyed, the first step is to get a new machine or commandeer an existing one. Then that machine is booted not from its local hard drive, but from a boot partition on the large capacity firewire drive. I keep a small (10G) boot partition on each drive and make sure that the OS revision is low enough to boot any machine I want to recover. Now I can use Disk Utility to replace the machine's main hard drive contents with the contents of the sparse disk image I select (as long as I remember the password of course). Several hours later I can reboot from the internal hard drive of the new machine and I am back in business.

I have had two hard drives fail on me (before I adopted this strategy) and lost very little because of my back up paranoia. So I know there are other ways to achieve effective back up. However it took days to recover each time, and was a painful and disruptive process.

Bagelturf Blog: http://homepage.mac.com/bagelturf

dnanian 06-11-2006 01:58 PM

Thanks for sharing, Bagelturf!

edoates 06-12-2006 06:38 PM

The only issue with this, from my own recent, painful experience, is that there are drive failures which will let SuperDuper run long enough to basically corrupt your backup leaving the message that the backup is in an unknown state.

My machine was using a strategy similar to the one noted above. My internal hard drive failed so that OS X (10.4.6, btw) would run, but there were lurking bad blocks. Finally, one was in a bad enough spot that SD failed (red alter on the status screen), and my backup sparse image would not mount properly; on reboot, the system would no longer boot (it had been up for a few weeks without rebooting, and even permission repair worked).

The solution is the noted more complex procedure (basically, treat the hard drive failure like a fire); or, depending on the size of the drive to be backed up, use multiple sparse images, smart backup, and a weekly (or so) cycle between the backups. A weekly verification that the last backup is good before cycling it would be needed, too.

Ed

PS: I was able to recover my data with Data Rescue II (neither DiskWarrior nor TechTool Pro 4 could help at all; DW spun forever trying to build a directory, TT4 crashed!).

JaneDoe 09-09-2007 01:23 PM

Quote:

Originally Posted by edoates (Post 7153)
The only issue with this, from my own recent, painful experience, is that there are drive failures which will let SuperDuper run long enough to basically corrupt your backup leaving the message that the backup is in an unknown state.

My machine was using a strategy similar to the one noted above. My internal hard drive failed so that OS X (10.4.6, btw) would run, but there were lurking bad blocks. Finally, one was in a bad enough spot that SD failed (red alter on the status screen), and my backup sparse image would not mount properly; on reboot, the system would no longer boot (it had been up for a few weeks without rebooting, and even permission repair worked).

The solution is the noted more complex procedure (basically, treat the hard drive failure like a fire); or, depending on the size of the drive to be backed up, use multiple sparse images, smart backup, and a weekly (or so) cycle between the backups. A weekly verification that the last backup is good before cycling it would be needed, too.

Ed

PS: I was able to recover my data with Data Rescue II (neither DiskWarrior nor TechTool Pro 4 could help at all; DW spun forever trying to build a directory, TT4 crashed!).


How does one verify that a backup is good?


All times are GMT -4. The time now is 09:25 AM.

Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2024, vBulletin Solutions, Inc.