SuperDuper!

Infrant Expansion Sunday, March 18, 2007

The other day, I was pushing at the limits of my existing Infrant ReadyNAS NV setup, and needed to increase its size. Normally, this would be a huge project, but with the ReadyNAS it was incredibly easy to do.

You see, the ReadyNAS uses Infrant's proprietary X-RAID. X-RAID basically RAID 6RAID 5 (see comments, below) with the ability to dynamically increase the total size of the RAID as well.

So, not only will the ReadyNAS run with a single drive faiure (and hot-rebuild the drive), it can dynamically increase the size of the RAID set as well. So, all I had to do was:

  1. Buy four drives of the appropriate size. I went from four 250GB drives with a total size of about 700GB, to four 500GB drives with a total size of about 1.6TB.

    The reason you don't get "all" the space on the drives is because redundant information is spread across each drive that allows any drive that goes "down" to be replaced and rebuilt with no data loss.

  2. With the ReadyNAS on, and in use, pull out the first of the four drives.

    Yeah. Scary. But that's what to do!

  3. Unscrew the four screws that attach the SATA drive to the tray from and attach it to the new.

  4. Slide the new drive into place.

    At this point, the ReadyNAS will automatically rebuild the data that was on the original drive on this new drive. All of this has been done with the unit on and operating.

  5. Wait for the rebuild to complete (it'll send you email when it's done).

  6. Repeat with the next drive.

Yeah. That's it. When you're done, you do need to restart the ReadyNAS to get the volume to expand, but that can be postponed until you're ready to do it... and that's the only time the unit is "down".

Pretty cool, eh?

(Yeah, I know I sound like a pitchman for Infrant, but I'm honestly not affiliated with them in any way at all. I just think it's a great product.)

Good-bye, iSight Saturday, March 17, 2007

Many of you probably know that, for some reason, iSight cameras can get into a weird state where they start causing serious errors with FireWire drives.

The symptoms usually include a bunch of I/O errors while copying to a FireWire drive: failed copies, flaky behavior, crashes. All of this goes away if you power off, disconnect the iSight, wait a while and power back up.

Sometimes, the iSight stops working with iChat (it says the camera's in use when it's not, or the light comes on, but you don't get any picture), and in it was in that second state today when I had a kernel panic.

No data was lost, I'm happy to say, but I decided that I've had enough of this. Since 10.4.9 supports USB cameras, I've switched to a Logitech QuickCam Ultra Vision. (All they need to do is add Super Deluxe Extreme Edition to the end to make it really cool!)

It's not as elegant as the original iSight (it's fixed focus, horizontal format, a bit gaudy), but it's got a nice wide angle lens, built in microphone, works with any monitor, and seems reliable. Plug the thing in, and it works.

Given that you can't buy "real" iSights any more, and based on direct experience, I give it a thumbs-up.

Sorry, iSight. I'll miss you!

Airport Disks Friday, March 16, 2007

When the new Airport Extreme Base Station came out, I was happy to see that it had some support for attached USB drives. It's a reasonable (though not perfect) way to share data in a home/SOHO situation, and I figured people would be using it to back up with SuperDuper! So -- to help out...

General Comments

A few things to point out right off the bat:

  • As you've likely seen elsewhere, the Airport Disks are not fast. Don't expect blazing speeds: wired, I've seen a maximum of about 1.5MB/s.

  • The AEBS gets very, very cranky if you get to a disk full situation. I've seen it crash more than once. Don't do that.

  • Remember that HFS+ drives are made available through AFP, and FAT32 through SMB.

    Don't format any drives you're going to use with SuperDuper! as FAT32: use HFS+ (and partition properly for the Mac processor type you're using -- GUID for Intel, Apple Partition Map for Power PC).

  • Make sure to connect a power supply to the drive.

  • As general advice, please don't cheap out when you get an external drive. Really. Get one with a real Oxford chipset, USB/FireWire if possible in case you want to attach it directly to your Mac.

  • Don't expect miracles. This is an inexpensive solution, and it behaves like one. If you want a real NAS, I suggest an Infrant ReadyNAS NV+: it's faster, redundant, recoverable. I'll have another post about the ReadyNAS soon.

  • Remember, this is a first generation ("Rev A") product. It's likely to go through teething pains. Don't rely on them as your only backup!

Using Airport Disks with SuperDuper!

You'll note that your Airport Disks don't show up in the SuperDuper! pop-ups. This is by design: we can't currently copy directly to or from a network volume due to authentication/permission/metadata issues.

Instead, you'll follow the steps in Backing up over a network in the SuperDuper! User's Guide (Help > User's Guide), and back up to a read/write sparse image stored on the Airport Disk.

We often get asked why SuperDuper! can't back up directly to a network volume. What most people don't realize is that, for security reasons, you can't directly authenticate as "root" over a network, and that means it's not possible to store files with system ownership on a network drive.

An image, on the other hand, acts as a "local drive", and can be authenticated against, even though it's stored remotely. This ensures that your files are backed up with full fidelity, including proper ownership and permissions. And since it's formatted as HFS+, it avoids various situations that can ensue trying to emulate HFS+ semantics and metadata storage on a non-HFS+ drive, while still storing in a native, non-proprietary, Mac-native format.

I'd suggest doing your first full backup directly to the USB drive, rather than over the network. This'll be a lot faster. You can then connect the drive to the base station, and re-select the image using the "Disk Image..." choice in SuperDuper!'s destination drive pop-up. (Note that although the image will be grey, you can still pick it, and ignore the "overwrite" warning. Yes, I know that UI sucks.)

That should do it: enjoy the base station!

Winter Sporting Sunday, February 04, 2007

Hopefully no one out there noticed, but—for the first time in three years—Zabeth and I managed to get away for a two week vacation at Red Mountain (in Rossland, BC).

It’s not too bad a trip, about three hours north of Spokane, WA, a straight shot after a (normally) easy flight or two. A few screw-ups this time meant various additional hops, but in the end we made it.

It’s my very favorite place to ski. It’s kind of an “old fashioned” ski hill, with four “low speed” chairs, a T-Bar and a (new) “magic carpet” lift. The base lodge is quite basic, with lockers in the bottom level, a cafeteria on the 2nd and a bar on the 3rd. Up in the “Paradise” area, there’s another small lodge/warming hut/eatery… and that’s pretty much it.

And, honestly, all that works great. The food’s good, the people are great. But what Red’s about is the skiing.

The skiing is awesome.

Red had a ton of early snow—about nine feet—and though we didn’t get much fresh snow in the two weeks the mountain was in fine form. The first few days were almost spring-like conditions on the front face, soft even early in the day. Red’s a deceptive mountain, though, where you can ski 360 degrees around the various peaks, with a ton of off-piste skiing (some of which can get quite extreme). And the backsides, even during the sunny days, stayed shaded, the snow light.

Which was a good thing, because once the temperatures dropped, the front firmed up, which made the less groomed, exposed trails much less enjoyable early in the day. So, after a few groomers, we typically headed to the various back glades, bowls and other pitches, finishing up in the sun at the end of the day.

We stayed at Greene’s Family Guest House, a great little place in Rossland run by Rick and Sue Greene (thanks for the hospitality, Rick & Sue). It’s a few minutes’ drive to Red itself, and—fortunate for me—they’ve recently installed a wireless network. So, via the Miracle of the Tubes, work from 7-9am, skiing from 9-3, work from 3-7, dinner, and work until bed (usually while we caught up with episodes of Lost we’d missed, thanks to an iPod full of episodes plugged into the TV).

Seamless, I hope, for the SuperDuper! users who needed help.

It was a great time, and quite relaxing despite having full work days every day, and I feel fortunate to have a job that lets me do what I need to do even when far from home.

My parents were also able to take Taiko for the time, and I hope he was well behaved while there. They seem to have done really well, and Z and I are really grateful that they could watch him. (Sorry that he’s still jumping!)

Got back late Friday (during a snowstorm, of course), and picked up Taiko on Saturday, who was happy to see us and is warming my feet as I type. The mail revealed that Zabeth has passed the Veterinary Boards, which is terrific news too (not that I had any doubt), and she heads back into rotations early tomorrow morning.

And so, it’s back to the grind—refreshed.

An Embarrassment of Eddys Tuesday, December 12, 2006

Wow.

It looks like SuperDuper! 2.1 has won another Macworld Eddy! Back-to-back awards for SuperDuper! in 2005 and 2006, and one for netTunes in 2004w00t!

Thanks, Macworld—and thanks to all the users, too!

The software business is a strange one, because the products you make eventually just… vanish. The OS goes away, or the market moves on, and soon there’s nothing left to show when someone asks what you do, or what you did.

Getting these three Eddy awards—one for netTunes and two for SuperDuper!—has been really gratifying, both because it’s an acknowledgment of good work, and because it’s more of a permanent thing—a physical record of what I was doing for all those years.

Plus, the trophy doubles as a weapon in an emergency. Man, these things are heavy!

Older, wiser. But mostly older. Thursday, November 23, 2006

Well, one year ago today we released SuperDuper! 2.0, and the past 12 months have gone quickly indeed.

In that time we’ve released a number of great updates, and I’ve communicated personally with thousands of you through support mail, IM, the forums and this blog. It’s been fun, rewarding and—on occasion—exhausting. But, mostly, fun and rewarding.

Looking over my blog post from this day last year, I spent my entire birthday, save for about 40 minutes, responding to hundreds and hundreds of support messages as people asked questions about the new release, and Bruce and I tried to fix a rollout glitch or two. Quite a day.

This year, though, it’s much calmer, and I’m going to take most of the rest of the day off—pretty much my first in a few years—to celebrate Thanksgiving (& my encroaching decrepitude) with my family, aunts, uncles, cousins, Zabeth and Taiko.

While I’m doing that, I also want to raise a glass of Thanksgiving wine in a toast to thank all of you, out in the tubes that make up teh Internets, for your support, encouragement, criticism, and compliments. Bruce and I couldn’t do this without you!

Happy Thanksgiving!

Fall Wednesday, October 18, 2006

No doubt you’ve noticed that I’ve been neglecting the blog a bit recently.

It’s been rather busy here, with various projects taking time as we hurtle through fall toward winter. Taiko’s quickly growing into his paws, and while it’s kinda silly to compare his personality to Ketzl’s, he seems to be a bit more mischevious, and he’s certainly a lot more willing to get up on his hind legs. It’s taking a lot of time to supervise him, correct him when he takes things off counters (or jumps on people) and get him the exercise and socialization he needs. But things are going nicely: he’s about 50lbs and shaping up to be a good boy. More photos soon.

Zabeth’s fourth year of veterinary school is going by quickly as she prepares to take the boards while, at the same time, doing her clinical rotations. She’s running on coffee and adrenaline at this point, and Taiko and I are trying to stay out of the way.

On the Shirt Pocket side, things have been busy. The release of 10.4.8 brought with it what looks to be a bug in Core Graphics: many applications—including SuperDuper!—are crashing on some Intel Macs when two threads are trying to draw at the same time. This happens in a lot of cases, but in ours we have some NSProgressIndicators that use the standard option that runs them on their own thread. If we’re updating the status view (in our main thread) at the same time the progress indicator tries to update, CoreGraphics uses a lock to handle the contention… but crashes.

Of course, it’s intermittent due to the timing issues, which makes it frustrating, but we’ve reported it both through the standard methods (rdar://4789778) and through other channels. We’re looking at workarounds here, since it’s unlikely 10.4.9 would come out based on this one problem.

For testing purposes, I brought a Mac Pro into Shirt Pocket Headquarters, it’s proven to be an excellent Mac. It’s very fast (although its I/O to a striped RAID set is much slower than I’d expect), very quiet and—so far—reliable. My few Boat Anchor applications are running beautifully in Parallels Desktop now that their MacPro compatible version is out—in fact, it continually surprises me how well Parallels works. If you need to run Windows, and don’t need high performance graphics, it’s a highly recommended solution. (Just make sure your VM is shut down before you back it up, of course!)

More as I get time!

Time’s Arrow Saturday, August 26, 2006

OK! netTunes and launchTunes release (and netTunes re-release—sorry about the Purple Rain) done, so it’s time to get back to what I keep getting asked about: SuperDuper! and Time Machine.

To get the “Frequently Asked Questions” out of the way right at the top of this post: no, we’re not dead, we’re not angry, and Apple has no obligation to leave market opportunities for independent developers, notify us that things are coming, or pretty much anything else.

This is business. It’s difficult for Apple to come up with 150 features to add into the next version of the OS, and harder still to make those features compelling enough that we’ll all pony up our hard-earned dollars to upgrade.

(As an aside, does anyone else out there think there was a definite hint, in the “feature” presentation of the keynote that bragged that OS X is now a “bigger, all-inclusive bundle”, that the price will be higher when Leopard is released?)

Some sort of backup functionality belongs in the OS. It’s been a long time coming. The fact that it wasn’t there left opportunities for 3rd parties, but that doesn’t mean Apple shouldn’t address the missing functionality.

And so, they have, with Time Machine. Really, I think that’s a great thing. People need to back up more often, and I hope Time Machine encourages them to do so.

Now, I can’t really get into a lot of details, because our NDA prevents disclosure of anything that wasn’t in the keynote. But let’s talk about what we’ve seen there, and why SuperDuper! remains both relevant and necessary—a true complement to the functionality in Time Machine.

First, as is likely obvious, Time Machine is designed to provide automatic “temporal” backup (discussed in broad terms in the post The Ninety-Nine-Per-Cent Solution many months ago).  Its primary usage scenario—and the one that the keynote focused on—is to allow quick recovery of files and data that have gone missing, etc. It does this in a way that’s highly integrated with the OS, with a unique UI that’s both cool and kinda cheesy… and, as was the case with Spotlight, with a certain amount of application-level impact (something 3rd parties like Shirt Pocket could never mandate).

What’s important to note is that this isn’t, and never was, what SuperDuper! was designed to do.

Our tagline, Heroic System Recovery for Mere Mortals, tries to sum up the whole idea: SuperDuper! is designed to provide excellent failover support for the all-too-common case where things fail in a pretty catastrophic way, such as when a drive fails, or your system becomes unbootable. We do this by quickly and efficiently creating a fully bootable copy of your source drive. Perhaps more importantly, recovery is near immediate, even if the original drive is completely unusable, because you can start up from your backup and continue working.

You can even take your backup to a totally different Macintosh, start up from it, and work while your failed Macintosh is in the shop… then, when it comes back all fresh and shiny, restore things and keep working.

All of this is done with a minimum of fuss and bother, and with respect for your time. And while Time Machine can restore a full system (the details of which were not shown, so I can’t comment on them), as can other similar products, that’s not its strength. Doing so requires you to actually take the time to restore the backup in full, which interrupts your workflow, requires a destination device, and takes a lot of your time—at the exact moment when you can least afford it.

So, when Leopard comes out, and Time Machine is released, be assured that we’ll continue to be relevant and necessary. We’ll work alongside its rapid recovery of individual files, and will seamlessly augment that with our rapid system recovery.

And, of course, we’ll continue to improve every part of SuperDuper! to make backups faster and easier for all.

(Digg this post.)

Backslash in AppleScript - Japan Style! Saturday, June 24, 2006

A few days ago, Shigeru Harada of MacFreak contacted me with a strange problem: he was unable to schedule backups with SuperDuper. Every time he tried, he’d get an error that the script wouldn’t compile.

In the past, we’ve seen this when drives were named with slash characters, but his were quite normal.

After some back-and-forth, I had him take the script template we use, and try to compile it himself, in Script Editor. Shockingly—it failed to compile, and the problem didn’t make any sense.

If you’re familiar with the Japanese keyboard, the backslash key () is replaced by the symbol for the Yen (¥). Way back when, we did a Japanese version of BRIEF, so I was familiar with this phenomenon—paths would be separated by Yen symbols, but everything worked as expected.

But, in AppleScript, it seemed the backslash/yen “swap” completely prevented backslash from doing its normal thing, so this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]"~`^\|*()”

completely failed to compile, because it looked like this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]¥"~`^¥¥|*()”

and ¥ didn’t escape as you’d expect.

A huge surprise to me. I did find it discussed in one place (thank you Google and Takaaki Naganoya), and also a reference to bug fixes in Tiger (Backslash characters and Yen sign characters will now compile correctly when the primary language is set to Japanese. [3765766]), but it didn’t seem to be fixed for him—probably because he’s using Panther (which we need to support).

Anyway, workaround in hand, I modified the script to:

set quoteChar to ASCII character 34
set backslashChar to ASCII character 92
set the URL_A_chars to “$+!’,?;&@=#%><{}[]” & quoteChar & “~`^” & backslashChar & “|*()”

Here’s hoping it works… I hope it’s not a problem with chevrons («») too, because I have no idea how I’d work around that one…
I/O Error Recovery Monday, June 19, 2006

I read Wolf’s I/O error treatise this morning (as well as Alaistair’s response), and thought I’d write a bit about how SuperDuper! actually handles I/O errors, and why. (In fact, this is an expansion and reworking of some email I dropped to Jonathan after reading the article.)

Although Wolf says otherwise, ditto isn’t our underlying engine. We use a variety of APIs in Cocoa and Carbon, augmented with much additional metadata copying. However, when we get a failure with those (such as an I/O error), we retry twice more: once with copyfile and once—just in case—with ditto, verifying after each one.

We do this because we’ve seen the rare case where one API fails but others do not. Weird, I know, but it happens.

If all three retries fail, we stop. This is done for safety: an I/O error could mean the drive is failing, and since you’re dealing with a live backup, it’s important to understand what’s going on. If a significant failure is occurring, steps should be taken to concentrate on recovering your user files, rather than trying to copy the whole drive. We don’t want a user (or SuperDuper!) to continue past the failure: we want them to stop, diagnose and—if necessary—get help. And since most users won’t know what to do (unlike Wolf or Alaistair, clearly), we make it really easy to contact support.

Our User’s Guide has a Troubleshooting section that helps a user determine whether the error is on the source or destination (I don’t explain there how to use the system log and a System Profiler report to locate the source, because it’s pretty obscure stuff—the amount of detail in our log is confusing enough for most), as well as general steps for recovery. But in most situations, 4K of 0s will pretty much be a fatal problem for the file. (I’m shocked, frankly, that Wolf’s Parallels disk was OK given the damage: he was very lucky.)

Most of the time, the problem is actually an iSight camera, iPod, or other bus-powered device misbehaving on the FireWire bus. On occasion, the problem is with the source.

Errors on the source are problematic. As Alastair mentions, modern disk controllers transparently relocate sectors when errors occur. Real problems happen when the drive’s out of spares, or when the on-disk error correction can’t handle the failure. And at this point, the drive has probably been silently failing for a while.

In many cases, SMART status will flag a drive that’s failing badly—SMART Reporter, a nice bit of freeware from Julian Mayer, can give you an obvious warning when this occurs, or even run a program (like SuperDuper!) to do a quick backup of critical files. But, often, it won’t, and experienced guidance and advice is necessary to help people understand what’s going on.

Anyway, as Wolf’s article indicates, and Alaistair agrees, it’s very difficult to continue in a way that ensures data is preserved as much as possible. It’s hard to know what really happened without being there, and an automated fix isn’t guaranteed. So, we’re super conservative. And while it’s obviously labor intensive, we think injecting a human into the process at this point provides the user with the best outcome.

Page 8 of 14 pages « First  <  6 7 8 9 10 >  Last »