PDA

View Full Version : SuperDuper and Metadata


jofallon
06-25-2008, 01:46 PM
I use SD 2.5 as my primary backup (with TimeMachine going to another volume). Before Leopard, the general feeling was that SD was one of the few ways to get a backup with all the metadata preserved correctly. I've seen a few comments since Leopard that SD does not get quite all the metadata completely correct (e.g. Tidbits column of May 31). Are you guys presumably addressing this issue in some future update?

I have no great confidence in TimeMachine, and am hoping any flaw in SuperDuper is very minor.

dnanian
06-25-2008, 02:02 PM
While we are going to correct this in the next release, the report is a bit misleading because the "problem" isn't just minor, it's of no consequence whatsoever to most users.. and doesn't involve metadata at all. Your files, and your metadata, are being copied correctly.

Basically the "flaw" is that SuperDuper! doesn't copy "FIFOs" or local device files. This was actualy by design, because -- in a standard OSX install -- device files are automatically created at startup time. And FIFOs are kind of 'queues' for communication between programs (rarely used) -- that are created at startup, typically in /tmp.

In fact, neither one of these types of 'files' (which are really special directory entries with no contents) can be copied at all -- they must be recreated on the backup, kind of as placeholders, because they don't really have much meaning across a restart.

We had decided long ago to not copy them (because we determined it didn't really make sense to do so), but since a tool is out that tests against this case, and because people might misinterpret the 'failure' to pass that particular test as something important, we've reversed that decision and will "copy" them in the future.

Note that, in general, we feel it's far more important to be able to faithfully copy actual file structures such as hard linked folders (something that Time Machine uses, but isn't exclusive to Time Machine, and that does have meaning across a restart), real data files, metadata (EAs, forks, ACLs), etc -- some of which the test tools don't actually check.

Hope that helps to clarify the issue.

MJT Services
06-25-2008, 04:17 PM
I have used SuperDuper as my primary backup for years, and have always found it to be reliable, not only on workstations, but on Mac OS X Servers as well, until recently.

A client of mine upgraded their server to Leopard Server 10.5.3. Now all of the client computers that connect and work from this server can no longer perform searches on the server. Neither Spotlight nor Find commands work with files on the server. However, if they restart the server, they can all search again. The SuperDuper runs in the middle of the night, and that seems to be the point at which searching ceases to function.

Could this be the reason that restarting the server fixes the problem, and it is actually the SuperDuper that is causing the inability to search after the backup is performed? If so, is there a work-around available until the software is updated? Possibly a setting to be turned on or off?

Thanks

dnanian
06-25-2008, 05:42 PM
I can't think of anything that would cause searches to fail on Leopard Server. We're just copying files... do you get any kind of indication in the system log of something gone wrong?

jofallon
06-26-2008, 03:11 PM
Eh? why would you want to copy those? The next time the partition boots up, they'd just be created.

OK, thanks for the info.

dnanian
06-26-2008, 04:55 PM
Right. You wouldn't want to copy them. But, Backup Bouncer is testing against them, and people don't know what they are, so they're worried about it. And, unfortunately, the latest "Take Control" book is using Backup Bouncer's tests as a baseline, so a lot of people are seeing the results without really knowing what they mean, or what they are. (In fact, I'm a bit surprised Joe Kissell is publishing that failure as a flaw without really explaining that it isn't.)

In any case, as I indicated, we're going to 'fix' it.

davep
04-27-2009, 09:21 AM
Dave -- I just read the part of Joe Kissel's book that mentions Backup Bouncer results. (I'm a bit late to the party.) I trust your decision to NOT copy the useless stuff, but understand your desire for a BB A+ score. How much extra time are we talking in order to copy this un-needed metadata? If it's a chunk, then perhaps you could include a preference for people to decide whether or not to copy it.

As drives are typically reaching 500GB, 1TB and even 2TB sizes these days it's getting important to speed up the backup process. Perhaps you could include a preference for a "faster backup" that omits copying useless stuff?

Dave P.
Massachusetts

dnanian
04-27-2009, 10:20 AM
It would take zero extra time to copy FIFOs because -- as I indicated above -- there are no FIFOs on any "normal" user system.

MacMacken
12-29-2010, 08:55 AM
Are you still working on a perfect Backup Bouncer result?

According to http://blog.dr3do.me/post/2497118340/backup-bouncer-test-superduper-2-6-2-v85, SuperDuper! still fails some tests. CCC on the other hand seems to pass with flying colors: http://blog.dr3do.me/post/2498487074/backup-bouncer-test-carbon-copy-cloner-3-3-7.

Martin

dnanian
12-29-2010, 09:02 AM
Well, we had perfect results in previous versions of BackupBouncer, and they've changed BB since our release. We'll take a look.

sjk
12-29-2010, 02:34 PM
, and they've changed BB since our release. We'll take a look.
There's also Mike Bombich's Improvements to Backup Bouncer (http://www.bombich.com/groups/ccc/wiki/7ba51/).

dnanian
05-30-2011, 10:45 PM
You know, we've checked these results over again here, and SuperDuper! 2.6.2 passes all these tests without any problem:

579$ sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying: basic-permissions ... ok
Verifying: timestamps ... ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ...
Sub-test: on files ... ok
Sub-test: on hardlinked files ... ok
Verifying: finder-flags ... ok
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: [ on files ] ... ok
Sub-test: creation time ... ok
Sub-test: modification time ... ok
Sub-test: [ on locked files ] ... ok
Sub-test: creation time ... ok
Sub-test: modification time ... ok
Sub-test: [ on directories ] ... ok
Sub-test: creation time ... ok
Sub-test: modification time ... ok
Sub-test: [ on symlinks ] ... ok
Sub-test: creation time ... ok
Verifying: hfs-compression ...
Sub-test: decmpfs xattr ... preserved
Sub-test: UF_COMPRESSED flag ... set
Sub-test: file contents ... match
Sub-test: creation time ... ok
Sub-test: modification time ... ok
Sub-test: hard link inode ... ok
Sub-test: hard link decmpfs xattr ... preserved
Sub-test: hard link UF_COMPRESSED flag ... set
Sub-test: hard link modification time ... ok
ok
Verifying: hfs-compression_large ...
Sub-test: decmpfs xattr ... preserved
Sub-test: UF_COMPRESSED flag ... set
Sub-test: file contents ... match
Sub-test: creation time ... ok
Sub-test: modification time ... ok
Sub-test: hard link inode ... ok
Sub-test: hard link decmpfs xattr ... preserved
Sub-test: hard link UF_COMPRESSED flag ... set
Sub-test: hard link modification time ... ok
ok
Verifying: access-control-lists ...
Sub-test: on files ... ok
Sub-test: on dirs ... ok
Sub-test: on locked files ... ok
Sub-test: on non-inherited acls ... ok
Sub-test: on inherited acls ... ok
Verifying: fifo ... ok
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok


We're pretty sure the user didn't verify the bbouncer results properly, and didn't use "sudo" when running the verify.