Shirt Pocket Discussions  
    Home netTunes launchTunes SuperDuper! Buy Now Support Discussions About Shirt Pocket    

Go Back   Shirt Pocket Discussions > SuperDuper! > General

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 09-12-2009, 09:06 PM
avramd avramd is offline
Registered User
 
Join Date: Sep 2009
Posts: 2
Requesting "Ultra-smart" TM-based backup feature

Hello,

I would like to open a discussion about a possible new feature. Obviously the panacea that everyone wishes either TM or SD! could do is automatically maintain a bootable backup that is up-to-date within the hour.

I believe I just discovered how SD! could do it; I would like a feature where SD combines TimeMachine's backup data, with the inode comparison technique used by timedog, to "UltraSmart Update" to a bootable backup drive by only copying the files that TimeMachine has found changes in since the last time the feature was used on the same target drive.

timedog is able to detect changes between TM backups incredibly quickly because it simply compares inodes. Since TM makes hard links for directories that have not changed, timedog does not have to scan the contents of a directory if its contents did not change. The heavy-lifting of identifying changes has already been done by Time Machine.

Some timedog performance bites: 1.5 secs to identify the changes of a 70-change TM backup of a 2.8GHz MBP w/ 90GB used on the drive being checked), and 2 min to identify the changes of a 20,000-change TM backup of the same drive (this was after updating to iPhoto 9 & letting it update my photo library).

Another benefit of this strategy is that it should be able to run from a network TM backup (e.g. TimeCapsule), since the image file mounts locally as a true HFS+ partition - so the generation of the up-to-date bootable backup could happen on another machine - and you can even take your machine away since TM is already done with it.

Also, you wouldn't need to be this pro-active with SD! - You can just run it say weekly, and let TM do it's thin in the mean time. Then after a crash, you run it again to turn your week-old SD! backup into a <1hr old one.

One obvious caveat of this technique is that the *first* backup would itself have to be generated from Time Machine data, to ensure proper continuity. Rather than building that capability into SD!, it would probably make more sense to generate it using TM's own restore feature.
Reply With Quote
  #2  
Old 10-06-2009, 09:56 AM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,358
Send a message via AIM to dnanian
Actually, no: the work here is done by fsevents, not Time Machine, really. And improvements to scanning can be made in future versions of SD, but the more coupling we have to complex software systems (Time Machine, fsevents), the more likelihood there is that a flaw in those systems will impact all your backups, rather than just one.

In general, that's unwise. SD! tries to do things in a simple, straightforward way. It's easier to test, more reliable, and the only real downside is a bit of extra time taken. Given that, on my 3-year+ old system, it takes < 18 minutes (16 minutes last time) to update a backup of over 2.5M files to a regular FW400 drive, the time savings isn't worth the additional risk/coupling until those systems are very well established (and perhaps not even then).
__________________
--Dave Nanian
Reply With Quote
  #3  
Old 10-06-2009, 10:38 AM
avramd avramd is offline
Registered User
 
Join Date: Sep 2009
Posts: 2
Ok, that is a valid point. So let me paint the picture a little differently: How often do you backup with SD!?

Here is how my life currently looks w/rt backups: TM backs up hourly. SD! backs up daily, in the middle of the night.

Two weeks ago, I had a drive failure at about 5pm. I had been working all day. I didn't particularly remember what I had touched that day. Here are the options I had for recovery:

1) swap in the SD! backup drive and be up and running immediately, but with the task of manually pulling from TM everything I could think of that had changed (time consuming), and living with an unknown amount of lost data (risky)

2) ignore the SD! backup drive, and restore from the TM backup being certain that everything I have is as fresh as possible (time consuming but not risky)

I took option 2. But in thinking it through, I realized that I can not think of a single situation in which I would take option 1 (well except for if I had an HD crash at 3am that I knew for sure happened after my SD! backup had completed, but before I had done any other work that TM! caught).

What I really want is an automated way to take an SD! backup and bring it up to date w/rt the latest TM backup. It just doesn't seem reasonable to run SD! hourly - if it takes 16 minutes to run, that means 25% of the time, your computer is running a process that is either pegging the CPU or the disk.

I guess another way to describe what I think needs to exist is an offline utility that can repeatedly update an offline backup based on the latest TM run. Not couple the logic of SD! & TM, just use the TM logic alone, but in a different way. It's reasonable to say that this would be an entirely different utility from SD! - but one that is sorely needed, IMHO.
Reply With Quote
  #4  
Old 10-06-2009, 11:36 AM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,358
Send a message via AIM to dnanian
In general, I find it pretty easy to know what I've been doing in the past few hours, so if there are critical things I need to restore from another backup, it's relatively easy to find them. But this is a usability problem with Time Machine, most certainly - it doesn't handle "please restore everything from the last four hours" well at all.

But, as I said, I don't think this is, in general, appropriate or desirable coupling between systems. One of the main advantages of using two separate programs to back up is to have additional failure modes covered.

If your Time Machine backup gets corrupted (and a small corruption can have massive impact on a Time Machine volume), your SuperDuper! volume is there for you. Similarly, if you have some critical files you absolutely must restore from your Time Machine backup, that's there too.

Tying both together in any significant way compounds, rather than reduces, any potential errors that either might encounter alone.
__________________
--Dave Nanian
Reply With Quote
  #5  
Old 10-06-2009, 12:13 PM
JoBoy JoBoy is offline
Registered User
 
Join Date: Oct 2007
Posts: 67
Quote:
Originally Posted by avramd View Post
Ok, that is a valid point... So let me paint the picture a little differently: ...in thinking it through, I realized that I can not think of a single situation in which I would take option 1 (well except for if I had an HD crash at 3am that I knew for sure happened after my SD! backup had completed, but before I had done any other work that TM! caught).
Unfortunately, I've had failures of my system twice in the recent past from a graphics card defect. The first time, I, too, took option 1 for the same reason, but I took option 2 the second time because I was aware of what data would be missing if I took option 2. There is nothing like real-life failures to stimulate thinking along the lines of this thread. However, I agree with the view that tying TM to SD! poses practical problems. There may even be proprietary issues with Apple.

Although it isn't true, it seems like TM is constantly trying to back up data. I've learned to ignore it and continue on with my work despite the constant background noise of the nearby Time Capsule. (I network TC via GbEthernet rather than using an external Firewire drive.)

My wish list includes a Super Duper! that runs simultaneously with my work flow and provides an updated, hourly backup, but not the versioning that is TM's stock in trade. That way, there is room for both apps in the market. TM would continue in its slow, versioning way and SD! would provide a quicker and reasonably current restoration that I would nearly always prefer because it restores a bootable system to my main drive in a very simple, very quick manner. I would use TM to restore an individual file or folder that I had previously deleted.

I have a concern about restoring from any copy that was made immediately prior to a system failure. Can such a copy perpetuate early file corruption that later leads to a system failure? This may be more of a worry than a valid concern, but I'd prefer something not quite so current for that reason. Losing up to 1 hour's work doesn't bother me, but concern over perpetuating corrupted files does bother me. Again, is that a valid concern? Could a really recent backup be programmed to quarantine the files copied during the last few minutes before a crash so that a person could elect to use them or not depending upon the cause of the crash?
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
alternating backup snoopy67 General 11 04-04-2009 06:14 PM
Hard drive size doubled after backup.... sanketr General 7 08-02-2008 01:16 PM
Feature Request: Last Backup Date bobm General 2 04-24-2008 03:31 AM
Server drive won't mount after backup rhennosy General 1 11-09-2007 04:49 PM
(Zero-length) File caused SuperDuper to abort backup alancfrancis General 7 08-31-2005 11:42 AM


All times are GMT -4. The time now is 04:32 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.