SuperDuper!

Airport Disks Friday, March 16, 2007

When the new Airport Extreme Base Station came out, I was happy to see that it had some support for attached USB drives. It's a reasonable (though not perfect) way to share data in a home/SOHO situation, and I figured people would be using it to back up with SuperDuper! So -- to help out...

General Comments

A few things to point out right off the bat:

  • As you've likely seen elsewhere, the Airport Disks are not fast. Don't expect blazing speeds: wired, I've seen a maximum of about 1.5MB/s.

  • The AEBS gets very, very cranky if you get to a disk full situation. I've seen it crash more than once. Don't do that.

  • Remember that HFS+ drives are made available through AFP, and FAT32 through SMB.

    Don't format any drives you're going to use with SuperDuper! as FAT32: use HFS+ (and partition properly for the Mac processor type you're using -- GUID for Intel, Apple Partition Map for Power PC).

  • Make sure to connect a power supply to the drive.

  • As general advice, please don't cheap out when you get an external drive. Really. Get one with a real Oxford chipset, USB/FireWire if possible in case you want to attach it directly to your Mac.

  • Don't expect miracles. This is an inexpensive solution, and it behaves like one. If you want a real NAS, I suggest an Infrant ReadyNAS NV+: it's faster, redundant, recoverable. I'll have another post about the ReadyNAS soon.

  • Remember, this is a first generation ("Rev A") product. It's likely to go through teething pains. Don't rely on them as your only backup!

Using Airport Disks with SuperDuper!

You'll note that your Airport Disks don't show up in the SuperDuper! pop-ups. This is by design: we can't currently copy directly to or from a network volume due to authentication/permission/metadata issues.

Instead, you'll follow the steps in Backing up over a network in the SuperDuper! User's Guide (Help > User's Guide), and back up to a read/write sparse image stored on the Airport Disk.

We often get asked why SuperDuper! can't back up directly to a network volume. What most people don't realize is that, for security reasons, you can't directly authenticate as "root" over a network, and that means it's not possible to store files with system ownership on a network drive.

An image, on the other hand, acts as a "local drive", and can be authenticated against, even though it's stored remotely. This ensures that your files are backed up with full fidelity, including proper ownership and permissions. And since it's formatted as HFS+, it avoids various situations that can ensue trying to emulate HFS+ semantics and metadata storage on a non-HFS+ drive, while still storing in a native, non-proprietary, Mac-native format.

I'd suggest doing your first full backup directly to the USB drive, rather than over the network. This'll be a lot faster. You can then connect the drive to the base station, and re-select the image using the "Disk Image..." choice in SuperDuper!'s destination drive pop-up. (Note that although the image will be grey, you can still pick it, and ignore the "overwrite" warning. Yes, I know that UI sucks.)

That should do it: enjoy the base station!

Winter Sporting Sunday, February 04, 2007

Hopefully no one out there noticed, but—for the first time in three years—Zabeth and I managed to get away for a two week vacation at Red Mountain (in Rossland, BC).

It’s not too bad a trip, about three hours north of Spokane, WA, a straight shot after a (normally) easy flight or two. A few screw-ups this time meant various additional hops, but in the end we made it.

It’s my very favorite place to ski. It’s kind of an “old fashioned” ski hill, with four “low speed” chairs, a T-Bar and a (new) “magic carpet” lift. The base lodge is quite basic, with lockers in the bottom level, a cafeteria on the 2nd and a bar on the 3rd. Up in the “Paradise” area, there’s another small lodge/warming hut/eatery… and that’s pretty much it.

And, honestly, all that works great. The food’s good, the people are great. But what Red’s about is the skiing.

The skiing is awesome.

Red had a ton of early snow—about nine feet—and though we didn’t get much fresh snow in the two weeks the mountain was in fine form. The first few days were almost spring-like conditions on the front face, soft even early in the day. Red’s a deceptive mountain, though, where you can ski 360 degrees around the various peaks, with a ton of off-piste skiing (some of which can get quite extreme). And the backsides, even during the sunny days, stayed shaded, the snow light.

Which was a good thing, because once the temperatures dropped, the front firmed up, which made the less groomed, exposed trails much less enjoyable early in the day. So, after a few groomers, we typically headed to the various back glades, bowls and other pitches, finishing up in the sun at the end of the day.

We stayed at Greene’s Family Guest House, a great little place in Rossland run by Rick and Sue Greene (thanks for the hospitality, Rick & Sue). It’s a few minutes’ drive to Red itself, and—fortunate for me—they’ve recently installed a wireless network. So, via the Miracle of the Tubes, work from 7-9am, skiing from 9-3, work from 3-7, dinner, and work until bed (usually while we caught up with episodes of Lost we’d missed, thanks to an iPod full of episodes plugged into the TV).

Seamless, I hope, for the SuperDuper! users who needed help.

It was a great time, and quite relaxing despite having full work days every day, and I feel fortunate to have a job that lets me do what I need to do even when far from home.

My parents were also able to take Taiko for the time, and I hope he was well behaved while there. They seem to have done really well, and Z and I are really grateful that they could watch him. (Sorry that he’s still jumping!)

Got back late Friday (during a snowstorm, of course), and picked up Taiko on Saturday, who was happy to see us and is warming my feet as I type. The mail revealed that Zabeth has passed the Veterinary Boards, which is terrific news too (not that I had any doubt), and she heads back into rotations early tomorrow morning.

And so, it’s back to the grind—refreshed.

An Embarrassment of Eddys Tuesday, December 12, 2006

Wow.

It looks like SuperDuper! 2.1 has won another Macworld Eddy! Back-to-back awards for SuperDuper! in 2005 and 2006, and one for netTunes in 2004w00t!

Thanks, Macworld—and thanks to all the users, too!

The software business is a strange one, because the products you make eventually just… vanish. The OS goes away, or the market moves on, and soon there’s nothing left to show when someone asks what you do, or what you did.

Getting these three Eddy awards—one for netTunes and two for SuperDuper!—has been really gratifying, both because it’s an acknowledgment of good work, and because it’s more of a permanent thing—a physical record of what I was doing for all those years.

Plus, the trophy doubles as a weapon in an emergency. Man, these things are heavy!

Older, wiser. But mostly older. Thursday, November 23, 2006

Well, one year ago today we released SuperDuper! 2.0, and the past 12 months have gone quickly indeed.

In that time we’ve released a number of great updates, and I’ve communicated personally with thousands of you through support mail, IM, the forums and this blog. It’s been fun, rewarding and—on occasion—exhausting. But, mostly, fun and rewarding.

Looking over my blog post from this day last year, I spent my entire birthday, save for about 40 minutes, responding to hundreds and hundreds of support messages as people asked questions about the new release, and Bruce and I tried to fix a rollout glitch or two. Quite a day.

This year, though, it’s much calmer, and I’m going to take most of the rest of the day off—pretty much my first in a few years—to celebrate Thanksgiving (& my encroaching decrepitude) with my family, aunts, uncles, cousins, Zabeth and Taiko.

While I’m doing that, I also want to raise a glass of Thanksgiving wine in a toast to thank all of you, out in the tubes that make up teh Internets, for your support, encouragement, criticism, and compliments. Bruce and I couldn’t do this without you!

Happy Thanksgiving!

Fall Wednesday, October 18, 2006

No doubt you’ve noticed that I’ve been neglecting the blog a bit recently.

It’s been rather busy here, with various projects taking time as we hurtle through fall toward winter. Taiko’s quickly growing into his paws, and while it’s kinda silly to compare his personality to Ketzl’s, he seems to be a bit more mischevious, and he’s certainly a lot more willing to get up on his hind legs. It’s taking a lot of time to supervise him, correct him when he takes things off counters (or jumps on people) and get him the exercise and socialization he needs. But things are going nicely: he’s about 50lbs and shaping up to be a good boy. More photos soon.

Zabeth’s fourth year of veterinary school is going by quickly as she prepares to take the boards while, at the same time, doing her clinical rotations. She’s running on coffee and adrenaline at this point, and Taiko and I are trying to stay out of the way.

On the Shirt Pocket side, things have been busy. The release of 10.4.8 brought with it what looks to be a bug in Core Graphics: many applications—including SuperDuper!—are crashing on some Intel Macs when two threads are trying to draw at the same time. This happens in a lot of cases, but in ours we have some NSProgressIndicators that use the standard option that runs them on their own thread. If we’re updating the status view (in our main thread) at the same time the progress indicator tries to update, CoreGraphics uses a lock to handle the contention… but crashes.

Of course, it’s intermittent due to the timing issues, which makes it frustrating, but we’ve reported it both through the standard methods (rdar://4789778) and through other channels. We’re looking at workarounds here, since it’s unlikely 10.4.9 would come out based on this one problem.

For testing purposes, I brought a Mac Pro into Shirt Pocket Headquarters, it’s proven to be an excellent Mac. It’s very fast (although its I/O to a striped RAID set is much slower than I’d expect), very quiet and—so far—reliable. My few Boat Anchor applications are running beautifully in Parallels Desktop now that their MacPro compatible version is out—in fact, it continually surprises me how well Parallels works. If you need to run Windows, and don’t need high performance graphics, it’s a highly recommended solution. (Just make sure your VM is shut down before you back it up, of course!)

More as I get time!

Time’s Arrow Saturday, August 26, 2006

OK! netTunes and launchTunes release (and netTunes re-release—sorry about the Purple Rain) done, so it’s time to get back to what I keep getting asked about: SuperDuper! and Time Machine.

To get the “Frequently Asked Questions” out of the way right at the top of this post: no, we’re not dead, we’re not angry, and Apple has no obligation to leave market opportunities for independent developers, notify us that things are coming, or pretty much anything else.

This is business. It’s difficult for Apple to come up with 150 features to add into the next version of the OS, and harder still to make those features compelling enough that we’ll all pony up our hard-earned dollars to upgrade.

(As an aside, does anyone else out there think there was a definite hint, in the “feature” presentation of the keynote that bragged that OS X is now a “bigger, all-inclusive bundle”, that the price will be higher when Leopard is released?)

Some sort of backup functionality belongs in the OS. It’s been a long time coming. The fact that it wasn’t there left opportunities for 3rd parties, but that doesn’t mean Apple shouldn’t address the missing functionality.

And so, they have, with Time Machine. Really, I think that’s a great thing. People need to back up more often, and I hope Time Machine encourages them to do so.

Now, I can’t really get into a lot of details, because our NDA prevents disclosure of anything that wasn’t in the keynote. But let’s talk about what we’ve seen there, and why SuperDuper! remains both relevant and necessary—a true complement to the functionality in Time Machine.

First, as is likely obvious, Time Machine is designed to provide automatic “temporal” backup (discussed in broad terms in the post The Ninety-Nine-Per-Cent Solution many months ago).  Its primary usage scenario—and the one that the keynote focused on—is to allow quick recovery of files and data that have gone missing, etc. It does this in a way that’s highly integrated with the OS, with a unique UI that’s both cool and kinda cheesy… and, as was the case with Spotlight, with a certain amount of application-level impact (something 3rd parties like Shirt Pocket could never mandate).

What’s important to note is that this isn’t, and never was, what SuperDuper! was designed to do.

Our tagline, Heroic System Recovery for Mere Mortals, tries to sum up the whole idea: SuperDuper! is designed to provide excellent failover support for the all-too-common case where things fail in a pretty catastrophic way, such as when a drive fails, or your system becomes unbootable. We do this by quickly and efficiently creating a fully bootable copy of your source drive. Perhaps more importantly, recovery is near immediate, even if the original drive is completely unusable, because you can start up from your backup and continue working.

You can even take your backup to a totally different Macintosh, start up from it, and work while your failed Macintosh is in the shop… then, when it comes back all fresh and shiny, restore things and keep working.

All of this is done with a minimum of fuss and bother, and with respect for your time. And while Time Machine can restore a full system (the details of which were not shown, so I can’t comment on them), as can other similar products, that’s not its strength. Doing so requires you to actually take the time to restore the backup in full, which interrupts your workflow, requires a destination device, and takes a lot of your time—at the exact moment when you can least afford it.

So, when Leopard comes out, and Time Machine is released, be assured that we’ll continue to be relevant and necessary. We’ll work alongside its rapid recovery of individual files, and will seamlessly augment that with our rapid system recovery.

And, of course, we’ll continue to improve every part of SuperDuper! to make backups faster and easier for all.

(Digg this post.)

Backslash in AppleScript - Japan Style! Saturday, June 24, 2006

A few days ago, Shigeru Harada of MacFreak contacted me with a strange problem: he was unable to schedule backups with SuperDuper. Every time he tried, he’d get an error that the script wouldn’t compile.

In the past, we’ve seen this when drives were named with slash characters, but his were quite normal.

After some back-and-forth, I had him take the script template we use, and try to compile it himself, in Script Editor. Shockingly—it failed to compile, and the problem didn’t make any sense.

If you’re familiar with the Japanese keyboard, the backslash key () is replaced by the symbol for the Yen (¥). Way back when, we did a Japanese version of BRIEF, so I was familiar with this phenomenon—paths would be separated by Yen symbols, but everything worked as expected.

But, in AppleScript, it seemed the backslash/yen “swap” completely prevented backslash from doing its normal thing, so this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]"~`^\|*()”

completely failed to compile, because it looked like this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]¥"~`^¥¥|*()”

and ¥ didn’t escape as you’d expect.

A huge surprise to me. I did find it discussed in one place (thank you Google and Takaaki Naganoya), and also a reference to bug fixes in Tiger (Backslash characters and Yen sign characters will now compile correctly when the primary language is set to Japanese. [3765766]), but it didn’t seem to be fixed for him—probably because he’s using Panther (which we need to support).

Anyway, workaround in hand, I modified the script to:

set quoteChar to ASCII character 34
set backslashChar to ASCII character 92
set the URL_A_chars to “$+!’,?;&@=#%><{}[]” & quoteChar & “~`^” & backslashChar & “|*()”

Here’s hoping it works… I hope it’s not a problem with chevrons («») too, because I have no idea how I’d work around that one…
I/O Error Recovery Monday, June 19, 2006

I read Wolf’s I/O error treatise this morning (as well as Alaistair’s response), and thought I’d write a bit about how SuperDuper! actually handles I/O errors, and why. (In fact, this is an expansion and reworking of some email I dropped to Jonathan after reading the article.)

Although Wolf says otherwise, ditto isn’t our underlying engine. We use a variety of APIs in Cocoa and Carbon, augmented with much additional metadata copying. However, when we get a failure with those (such as an I/O error), we retry twice more: once with copyfile and once—just in case—with ditto, verifying after each one.

We do this because we’ve seen the rare case where one API fails but others do not. Weird, I know, but it happens.

If all three retries fail, we stop. This is done for safety: an I/O error could mean the drive is failing, and since you’re dealing with a live backup, it’s important to understand what’s going on. If a significant failure is occurring, steps should be taken to concentrate on recovering your user files, rather than trying to copy the whole drive. We don’t want a user (or SuperDuper!) to continue past the failure: we want them to stop, diagnose and—if necessary—get help. And since most users won’t know what to do (unlike Wolf or Alaistair, clearly), we make it really easy to contact support.

Our User’s Guide has a Troubleshooting section that helps a user determine whether the error is on the source or destination (I don’t explain there how to use the system log and a System Profiler report to locate the source, because it’s pretty obscure stuff—the amount of detail in our log is confusing enough for most), as well as general steps for recovery. But in most situations, 4K of 0s will pretty much be a fatal problem for the file. (I’m shocked, frankly, that Wolf’s Parallels disk was OK given the damage: he was very lucky.)

Most of the time, the problem is actually an iSight camera, iPod, or other bus-powered device misbehaving on the FireWire bus. On occasion, the problem is with the source.

Errors on the source are problematic. As Alastair mentions, modern disk controllers transparently relocate sectors when errors occur. Real problems happen when the drive’s out of spares, or when the on-disk error correction can’t handle the failure. And at this point, the drive has probably been silently failing for a while.

In many cases, SMART status will flag a drive that’s failing badly—SMART Reporter, a nice bit of freeware from Julian Mayer, can give you an obvious warning when this occurs, or even run a program (like SuperDuper!) to do a quick backup of critical files. But, often, it won’t, and experienced guidance and advice is necessary to help people understand what’s going on.

Anyway, as Wolf’s article indicates, and Alaistair agrees, it’s very difficult to continue in a way that ensures data is preserved as much as possible. It’s hard to know what really happened without being there, and an automated fix isn’t guaranteed. So, we’re super conservative. And while it’s obviously labor intensive, we think injecting a human into the process at this point provides the user with the best outcome.

Network Backup Drive Recommendation Thursday, June 15, 2006

Users often ask me for drive recommendations, and while I provide general suggestions for FireWire drives in the SuperDuper! User’s Guide, I haven’t suggested any Network Accessible Storage (NAS) devices, mostly because I haven’t been entirely happy with the ones I’ve tested.

In general, most NAS units are Linux-based, with a drive that’s formated as ext3, which allow a large sparse image to be stored on the drive. This works well with SuperDuper!, and lets us properly preserve permissions and metadata with full fidelity. Some, like the LaCie “mini” Ethernet drive, are shipped as FAT32, and need to be reformatted as ext3 (or HFS+, or NTFS) before they can be used.

I’ve tested the Buffalo Linkstation and Gigastation, the LaCie mini ethernet drive, the Linksys NLUS2, and a number of others, and while they all work reasonably well when configured properly, none have been recommendation-worthy.

The units I’ve tried in the past have all had very old AFP (Apple’s network protocol) support, old enough to not support files larger then 2GB or so (even though their file system could handle larger), mostly because they’re all based on an old version of Netatalk. SMB can be used instead, and that does work, but you have to know to do it… and most users don’t. This means their backups would fail in various weird ways.

On top of that, these NAS units are pretty insanely slow. Even the Gigastation, using Gigabit Ethernet, did an initial backup running at less than 1MB/s. This improved once the first copy was done, but it’s painful the first time through.

So… nothing to recommend.

Until now.

Over the last week or so, I’ve been doing extensive testing with the Terabyte version of Infrant’s ReadyNAS NV, and I’m very impressed: impressed enough to give it a big thumbs up.

The ReadyNAS linked above is a 4-drive X-RAID unit (which has four 250GB drives set up for fault tolerance and speed, using Seagate’s Nearline SATA drives which—like Maxtor’s MaxLine III drives—are designed to have a longer life and more heat tolerance than regular drives) in an attractive case, with GigE capability. Like its competitors, the ReadyNAS runs Linux, but unlike the others they’re fanatical about keeping things up to date, and support the most recent Netatalk, which properly handles large files.

On top of that, they’ve spent a lot of time optimizing the unit, and it actually delivers very good performance—I got about 5MB/s on a full copy-- far better than the others, even over AFP. (More AFP-specific optimizations are promised for the future, too.)

But the goodness doesn’t stop there. Infrant knows that their OS can do more than just serve up shared storage, and they’ve gone ahead and created a nice port of Slim Devices‘ SlimServer, which runs on the ReadyNAS and can serve music around the house, through real Squeezeboxen or the Softsqueeze player. (In fact, Slim Devices is offering a bundle of a terabyte ReadyNAS NV and two wireless Squeezeboxen for $1499.) Infrant also supports UPnP streaming, etc, so it’s likely to work with the solution you already have.

And since they have good AFP support, you can point iTunes to its media share, and rip directly to it. Nice!

The ReadyNAS is not inexpensive (although the 250GB, easily expanded one isn’t too bad), and you might wonder whether you should just get a Mac mini with a FireWire drive instead. While you certainly could do that, the RAID capabilities built into the ReadyNAS are invaluable when you’re dealing with critical data. Infrant has clearly designed their unit to be rugged and reliable, and that’s obviously important. A failing/failed drive can be hot swapped for a new one, so your data remains safe even in the event of that inevitability.

The only real downside of the Infrant—other than the expense—is that it’s got a fan. Heat is the enemy of hard disks, so it’s important for the fan to be there, and they’ve made it temperature sensitive so it only runs as fast as it needs to. But, it’s not silent (although it’s not that bad), so if that’s an issue, this isn’t the unit for you.

Otherwise, a very enthusiastic recommendation: Infrant has done some really great work here.

SuperDuper 2.1.2 now available Tuesday, June 13, 2006

Have at it, everyone!

Page 7 of 13 pages « First  <  5 6 7 8 9 >  Last »