non-heroic recovery FAIL! what should i have done?

[sumguy]

hi all -

i thought i was covered for backups, using both super duper and time machine. but when the Unfortunate Event finally happened, it seems i messed it all up! i don't feel heroic at all!

i'd like to relate the story of what my brilliant plan was, and how it all went wrong, in the hope that other people can avoid my fate, and that someone might give me some advice on how to do better next time...

i have an external usb drive with two partitions - one has my automatic hourly time machine backup, alongside a bootable copy of the snow leopard install disc. the other partition has a weekly bootable backup made with super duper.

my plan was that if my mac died, i could boot another mac from the super duper backup partition, and use time machine to update it to the latest state, i.e., an hour or less from the time the mac died. then i could continue working, and when my mac came back from the repair shop, i could use super duper to copy the working partition back to the internal drive.

well, the first part of the plan worked - my mac died! after that, i was able to boot another machine from the super duper backup. but then everything went off course...

i couldn't figure out any way to use the time machine backup to update the super duper backup partition. i assumed you could do something like a "smart update" with time machine, i.e., update only the files that have changed, and make the drive match the last saved state in the time machine backup. but all i could see was either to do a full restore, or just manually restore files one by one. considering i have something like a million files on my mac, there's no way i could remember everything that had changed in the past week, since the last sd backup! did i miss some option in time machine?

so then i decided to forget about super duper, and do a full restore from time machine. i booted from the copy of the install disc, erased the super duper backup partition, and restored the most recent time machine state to it from time machine. it took several hours, but finally i had a copy of the drive exactly as it had been, just before my mac died - or so i thought.

when i booted it, the first thing i noticed was that safari had lost all its page preview thumbnails on the bookmarks page. then i tried mail, which said something like "in order to use the new features, mail must import all your mail messages". and then time machine said something like, "if you make a backup now, you won't be able to use it on your other computer". obviously this was not an exact copy of my drive! i wondered and worried about what else was missing or not properly restored, and about screwing up my previous time machine backups.

the hardware problem with my mac wasn't related to the hard drive, the data on it was still ok. so in the end i just gave up. i waited for it to come back from repairs. i didn't feel comfortable about erasing it and replacing it with what had been restored from time machine, and i had already lost the sd backup. and actually i felt kind of stupid and frustrated about the whole thing. thankfully i didn't lose any data, but i lost a few days of my time, and it turned out my backup plan was a failure.

so, live and learn - what should i have done? doing hourly super duper backups isn't practical. but going back to a several-days-old backup doesn't seem really sensible either, when i also have an hourly time machine backup. did i miss some way to better integrate them? and, what's the deal with time machine full restore? is it just not reliable?

i realize that very little of my sad story can be blamed on super duper. it did exactly what it said it would do, and i do appreciate that. but the reality is that time machine exists, and in most cases will have a more up-to-date copy of your files. when you throw that into the mix with super duper... well, i for one failed spectacularly at implementing a successful workflow. anyone else?

cheers!

[dnanian]

Wow, I certainly wouldn't have overwritten the entire backup with Time Machine: you kind of had two fallbacks, and you destroyed one.

What I would basically suggest, in the event of a major disaster like you had, is restoring the SuperDuper! backup (a weekly isn't, in my opinion, often enough unless you also have a daily, but it's better than nothing) and "cherry picking" the Time Machine backup for the contents you were working on. That is, restore the Documents folder, perhaps your Desktop (you can do entire folders at once), and anything else absolutely critical.

Of course, you can restore the Time Machine backup in full but, as you saw, that doesn't always work out as "magically" as advertised. (You can have problems with any backup, of course, but you can't really "test" a full Time Machine restore conveniently, whereas you can boot from a SuperDuper! backup to see what you'll get.) So, don't overwrite your "good" (but a bit older) backup with an unknown one, at least not in full, unless you're willing to lose data that is now suddenly in only one place again.

Does that make sense?

[sumguy]

well, at the time i overwrote the super duper backup, i was pretty frustrated!

if i had lost the original drive, i would have done things differently. but i already knew the drive on my broken mac was still ok. i just had to wait several days for it to come back from the repair shop.

in the meantime, my goal was to have an exact copy of the drive, up to the time of the last hourly backup, to continue working, and then erase my mac's drive when it came back and transfer the working copy back to it. but since i was unable to find any method to obtain that exact copy - even with a full bootable backup plus hourly time machine backups! - i wasn't willing to erase the original drive. therefore the two backups i had were somewhat useless, and destroying one of them wasn't such a big deal.

if lost the drive now though, i still don't know what i'd do. i still don't feel that i have a workflow i'm happy with.

the manual "cherry picking" relies too much on my poor memory... what was i working on exactly? did i install or update any software? change any system or user preferences? yes, it would be better if i could manage to do daily super duper backups, but even then... it's frustrating to know i have (in theory) a full hourly backup, but am unable to update from it in an automated way. furthermore, the main "absolutely critical" thing for me is e-mail, and i'm now very skeptical about time machine's ability to restore it.

of course, the main frustration is time machine. i was quite surprised and disappointed to find out that a) there's no "smart update" type of feature to make your drive look like the backup, only a full restore that takes several hours; and b) that the full restore just doesn't seem to work right.

some people have said it's "normal" that mail has to re-import everything after a system restore. but apple's tech support pages don't say anything about it, only "follow the directions on the screen". i guess i should be talking about this in apple forums, but i'm also interested in feedback from people here, whether anyone else has managed a better solution.

thanks for your time and advice, it's appreciated!

[dnanian]

I can't really help you with your Time Machine frustrations, unfortunately: it's something you'll have to take up with Apple. And what I suggested is really the best I can do. But you don't have to rely on your memory, really - you can use a smart folder in the Time Machine interface to show all the files changed in the past week, say, which would narrow things down significantly...

[sumguy]

thanks dave, for your reply - the suggestion to use a smart folder in time machine to find changed files since the super duper backup is great, i hadn't thought of that.

and sorry to talk so much about time machine, it's not your fault!

i'm still interested to hear if anyone else has opinions or other suggestions about doing a full restore with super duper vs. time machine, or a combination of both... thanks!