SuperDuper!

30-day notice of awesomeness Tuesday, September 04, 2012

We've been pretty happy with the way v2.7's been behaving out there in the Big, Beautiful World, but as my previous posts indicated there were some bugs that we needed to deal with.

And deal we have: I'm happy to announce the v2.7.1, available today, has resolved the vast majority of them:

  • We've fixed the "too many open files" error, and the "index out of bounds" exception. While we were doing that, we improved our error handling even more, and added additional, more specific diagnostic messages during a failure that will help you (and us) pinpoint the cause of disk errors when they occur.

  • The statistics in the "status view" and in the log have been corrected. Previously, they were underreporting the number of files and bytes copied in some situations (even though the files were being copied as they should have been).

  • 10.4.11 users had a minor problem with Growl notifications that has been dealt with, too.

A fix for automatic mounting under 10.8 is still in progress (it's not a simple fix, even though we know what to do).

That'll do it for now—we'll head back to the code mines to work the remaining seam while you surface dwellers enjoy.

Hope you had a great Labor Day weekend!

Talkin’ about 2.7’s rollout Thursday, July 26, 2012

Two blog posts in two days - shocking!

The rollout of 2.7 is going well: this is probably the smoothest new version release we've had. But, of course, when you go from "small batch testing" to "large batch testing", you find some things that slipped through the cracks.

Here's what we've found and what we're working on - save for the first two, these are all extremely rare (fewer than ten users affected by all the issues combined), but I thought they might be interesting:

  1. As mentioned in the previous blog post, automatic mounting of ejected local volumes doesn't work under Mountain Lion when time-triggered schedules run. A fix is still in process.

  2. Some users on 10.5 and earlier are getting errors during the update process. If the update doesn't install automatically, you can download it from the Shirt Pocket web site and install manually.

  3. We've had a few reports of a "too many open files" errors in the log. This seems to have to do with some copy-retry logic for busy files. Investigation continues: if you're encountering this (unlikely), quit all active applications and retry the copy.

  4. Another three or four users are getting an exception with an "index out of bounds" error. We're pretty sure this is related to folders whose case has changed on a case-insensitive volume: we're trying to optimize for that, and update the case of (rename) the folder on the destination, and in some situations this can generate an error. To work around the problem, do an erase-then-copy backup rather than a Smart Update (one time).

  5. Finally, there's some kind of issue with utilities that mount volumes outside of the /Volumes folder (again, a very rare case): we're trying to descend into that mount, which will often generate an error. If this is happening to you, you can ignore the mount point/folder with a copy script by following the steps in the User's Guide.

That's about it! We'll continue feeding Xcode some Zwiebacks to get these few teething problems taken care of and get another update out as soon as we're done.

Silence is golden, but it’s time to talk Wednesday, July 25, 2012

The worst part about OS updates is that we can't talk about them.

I know other companies will sometimes discuss Apple's next version of OS X, and how they do or don't work with it, but we try to keep our mouth shut until it's actually out there and we're released from our non-disclosure.

That day is tomorrow, as I write this, and today, as I post it.

So let's talk about Mountain Lion and SuperDuper!

SuperDuper! is already Mountain Lion compatible

The version of SuperDuper! that was released last year, v2.6.4, was already compatible with Mountain Lion, with three exceptions:

  1. SuperDuper! v2.6.4 is not signed, so it will get flagged as a scary-bad application with the default Gatekeeper settings. It's not, of course, and already installed copies of SuperDuper! should work fine.

  2. Automatic mounting of ejected local volumes for scheduled copies does not work.

  3. Most of you can ignore this, but for the technically inclined: while SuperDuper sandboxes work, sandboxing of Mountain Lion (for future Mountain Lion updates) doesn't copy the applications Apple added to Mountain Lion (e.g. Messages, Notes, etc).

SuperDuper! v2.7 is more compatible with Mountain Lion. Plus it's even better.

I'd love to say that we've been sitting around sipping Corpse Reviver #2s and relaxing full time since the last update. That would be pretty awesome, but, well, no.

Instead, we've been working to make SuperDuper! better in a bunch of significant ways, which include:

  • Faster file copies That's right, your backups will finish faster than before.

  • Better information during copying We now update the status window while large files are being copied, so you can get a better idea of what's going with your backup while you sit back, relax, and have a tasty beverage of your own.

  • Gatekeeper compatibility SuperDuper! is now signed, and will not generate Gatekeeper errors when installed with Apple's default settings.

  • Much faster startup SuperDuper! starts even faster, even when you have unresponsive network volumes attached.

  • Better copying of active files Applications that rapidly create and delete files during a backup no longer cause intermittent "vanishing file" failures.

  • Better handling of Time Machine As much as we wanted to copy the local snapshot (the .MobileBackups folder), it was in an uncopyable state too often. It's now ignored, which results in fewer backup failures.

  • Improved diagnostics We've worked around problems in Lion's (and Mountain Lion's) file copy APIs, and more accurately return errors when drives can't be read or written.

  • Support for Growl's latest version & Notification Center We now support the version of Growl in the Mac App Store, which means we also support its latest features, including Notification Center.

  • Still supports Intel and Power PCs; OS X 10.4.11, 10.5, 10.6 and 10.7 As crazy as it seems (and as much as a pain it's been), we still support versions of OS X released in 2005, and Macintoshes that that should have been made into fish tanks years ago.

    In other words: we, clearly, are not smart, and those of you with older systems benefit!

  • Various other miscellaneous improvements Because we can't help ourselves.

Wait a second. You said "more compatible", not "fully compatible"!

You spotted a weasel word! Well done! Nothing gets past you, clearly.

Automatic mounting of local volumes is the one thing we couldn't get to work in time for Mountain Lion's release. (Backup on connect still works just fine.)

Basically, Apple is tracking where an application launches from, and seems to be preventing processes that launch from "cron" (one of the system schedulers), even indirectly, from mounting local disk volumes, even when that application is signed.

We found this late in testing, and tried valiantly to try to work around it, but were unable to in time. And given that it's undocumented (but really cool) behavior, I just couldn't justify holding up the release of v2.7 based on this one problem.

Don't lose hope, though: we're think we know how to fix it—initial testing looks good—so we'll release another version of SuperDuper! as soon as we get the fix fully implemented and tested.

How do I get the new version?

Start SuperDuper! and it will prompt to update itself (unless you've turned that off), or install it manually from the download at the SuperDuper! page at Shirt Pocket.

Yeah, but how much does the new version cost?

That's always the way, right? Show me the money, etc.

Well, we've never charged for updates, not since SuperDuper!'s release in 2004. Not once. And we're not raising our prices.

The update is free. A SuperDuper! registration still costs $27.95.

And the unregistered version still never expires, will make full, bootable backups, and comes with support.

We know it's not the most profitable way of handling things. But as long as you—our users—continue to run, love and recommend SuperDuper!, we'll continue to do our damnedest to do right by you.

So there you go: thanks for reading this and for using SuperDuper. Enjoy the new version!

Maratona dles Dolomites and an Update Saturday, July 09, 2011

For SuperDuper! users, I'll cut quickly to the chase here: we should be releasing an update this weekend that resolves the few issues that inevitably arise when a new update comes out. Thanks for your patience as we worked through the issues, and thanks to the users who helped us by running special test versions to ensure we had the problems fixed.

This is a bit of a stressful time to be releasing an update, as I mentioned before: I'm in Italy, training for the Maratona dles Dolomites, which happens tomorrow.

My schedule has been kind of crazy, as everyone suddenly remembers they have to back up before a major OS release (and given the minor issues we've been dealing with): I get up around 5-6am, answer as much email as I can before 7:30am, join the group for breakfast and route review, pack (we were moving to different lodging every day before today), get on the bike, ride over mountain passes until evening, shower, answer email until 7:30pm, eat, and then work from 9-10pm until sometime after 1am.

And repeat.

Our total activity so far, pre-Maratona: 210 miles of cycling over about 18 hours and 31 minutes, with 26,813 feet of climbing. Tomorrow's event will add up to 85 more miles and 14,000 feet of climbing over a long day with thousands of other cyclists from all over the world, for a trip total of around 295 miles and more than 30,000 feet of up.

Now, for many people (and you know who you are) that's really not that much. But this is the first time I've done anything quite this strenuous, and combined with a full day of work it's been mentally and physically exhausting. But I knew what I was in for when I signed up: the riding's been well planned, guided and supported, and SuperDuper users have (for the most part) been pretty understanding, so thanks!

With that, on to more detailed information about the update. Assuming final confirmation tests go OK, it should come out today (Saturday). Full information about the update is in the release notes, but I wanted to highlight a few items here.

First, we've fixed the problem with Tiger (10.4) where most AppleScript-based actions didn't work, including scheduling and post-copy actions like Shutdown and Restart. As soon as we determined this was a problem, we turned off auto-updating for 10.4 users.

This problem was due to a misconfigured build script that compiled our dictionary for 10.5 and later rather than 10.4 and later.

It's getting harder and harder for us to maintain compatibility with Tiger, and doing so is preventing us from using the new APIs introduced in Leopard, so I expect that we'll continue with our policy of supporting two OS versions back with new releases of SD! when Lion comes out. We'll still provide support for Tiger users, of course, but new versions will not be compatible as we move forward.

Many Leopard users found that the v2.6.3 was generating errors with their system log, asl logs or some other 'active' files. As above, as soon as we received a few similar reports of this problem, we turned off auto-updating for 10.5 users until we could run down the problem.

It turned out that some new, more aggressive post-copy error checking was a bit too aggressive on 10.5, and when users had very active system logging (due to various system errors that were getting written multiple times a second), a failure to verify file size (etc) information post-copy caused us to raise an I/O error for the file.

We've loosened up our check a bit here to avoid this problem, while still doing additional verification to catch more problems under all OS versions.

So there you go: auto-update will be turned back on for 10.4 and later sometime today, and you'll be able to enjoy improved operation while I enjoy a rest day off the bike.

As always, thanks for your emails, support requests, registrations and comments!

Riding a Lion Thursday, June 30, 2011

There's a certain way I like to release a version of SuperDuper, or any application, really. It goes something like this:

  1. We plan a healthy mix of fixes, tweaks and features.
  2. Those new features are designed and implemented.
  3. Once we've put them through a bunch of internal testing, we recruit a mix of old and new testers to put things through the "customer wringer".
  4. The fix/test cycle is repeated as many times as is necessary to ensure that we're meeting our quality standards, that new features are working as desired and satisfying the needs we'd identified during their design, etc.
  5. During the last few testing cycles, I usually start telling the story of the release (which, at that point, we're pretty sure we're nearly done with), highlighting some of the more interesting elements.
  6. Finally, we freeze the release and, with any luck, we release shortly after the release candidate is OK'ed by internal and external tests.

And I highlight all this because the whole "telling stories" part of our next release is going to be much shorter than usual, and we'll be doing other things a bit differently as well.

Ring One–Personal

As anyone who follows me on Twitter is aware, I've been doing a lot of cycling over the past few years (and tweeting about it in a rather boring way, sorry). After a cycling trip to Italy last fall, and based on a suggestion by one of the guides, I decided to train for (and participate in) the Maratona dles Dolomites.

So, for the last six months or so—in sun, snow and rain—I've been riding and training (well, trying to train) for this huge, 9500-participant Grand Fondo. In one day, we'll be riding about 85 miles and over 7 big passes, with 14,000 feet of climbing. It's the first time I've ever tried anything this...crazy. Ready or not (mostly not), on July 2nd, I'm headed to Italy for the race, which happens on the 10th.

Those ten hard-date days were running through my head when the 2011 WWDC keynote was broadcast earlier this year.

Ring Two—Professional

As the various rumor sites could tell you, it's always hard to know when Apple is going to do things. But given that WWDC was likely going to be all about 10.7, and last year's WWDC was about iOS, it seemed a reasonably safe bet that the release of 10.7 would be well after this year's WWDC.

Well that was a bad bet.

Apple announced that Lion will be coming out "in July". In other words, the release window opens tomorrow (I'm posting this on June 30th). Given the current state of things, our current 2.6.2 release of SuperDuper is not Lion compatible. Specifically, we know we have two issues of significance under Lion:

  • Deprecated command-line tool The "disktool" command was deprecated and effectively removed, and we rely on that for a number of things, including refreshing disk mounts when a copy starts, and automounting during a scheduled copy.
  • Updater crashes Our automatic updater crashes under Lion, which makes it hard for Lion users to update (although they can download from our site and install manually).

With that in mind, also remember that Apple is going to be releasing this through the App Store. That means that there's no delay between an "RTM" build (which we can test against) and when you get the GM: it'll get declared RTM, someone will push a button somewhere and BAM! it's GM, and in the App Store.

Add to that the fact that I'll be gone nearly half of July. Which means, if Lion comes out in the first half of the month, as it very well could, and probably will...I think you can see where this is going.

Ring Three–Where the Rubber Meets the Road

Well, perhaps not where you think. We're not going to be late with Lion compatibility, unless something really crazy unusual happens between now and when Lion hits your Mac.

What it does mean is that some of the more interesting things we were going to put in this update will be postponed until later, and we're going to have to put this out with "initial" Lion compatibility, as much as I hate doing that.

So here's the plan: we're going to release version 2.6.3 of SuperDuper! in the next few days (probably on July 1st). This version has a lot of small things in it, some of which will significantly improve the user experience. Examples include:

  • Initial Lion compatibility As I've explained, Lion is not out, so we can't promise that we'll be compatible with it when it is released. But SuperDuper 2.6.3 works well with the test version of Lion that we have now, and should work fine when the Big Button is pressed.
  • Better scheduling A small change that should help a lot of users. Right now, it's too easy for a user to accidentally create multiple schedules. In the new version, the "Schedule..." button brings up an existing schedule for the source/destination drive pair if there is one (which is what most users expect it to do). Users can still create another schedule for the same pair of drives by Option+clicking Schedule. (I could write an entire post about this one change, and hopefully will be able to in the future, since it's one of the major errors I made in SD!'s quickly-reworked-after-the-original-disaster schedule design.)
  • Improved volume failure detection If the destination volume vanishes during the copy, we detect the failure and stop the copy more reliably than before. This should avoid cases where the destination drive's mount point converts to a folder and files are copied to a "hidden" location (/Volumes) on the startup volume.
  • Antivirus compatibility improvements We've tried to work around some incompatibilities with NAV that would cause intermittent issues for users that had that installed.
  • Better support submission We've had an integrated email support submitter for a long time, but it relied on the NSMessaging framework, which has been deprecated. So, we've moved to a "direct submit" model. You shouldn't see any differences locally, but it'll work in more situations, regardless of your email client. (The downside is that we can't 'automatically reply' to one of these messages, which means a lot more work for me...)
  • Expanded automount/ejection We now allow internal drives to be ejected, so they can participate in automatic mounting when scheduling, and can be selected as an ejectable volume after a copy.
  • Various other improvements, fixes and tweaks We've touched pretty much every area of SuperDuper! in some subtle (and some not so subtle) ways, with more to come.

Thanks for Coming—Visit the Midway!

So there you go: we'll soon have SuperDuper! v2.6.3 available. You should endeavor to upgrade to the new release before you upgrade to Lion.

I hope you like the new release! If you're interested, follow me on Twitter where I'll be posting about the Grand Fondo a bit, and will be posting updates about SuperDuper! as necessary.

Support during these two weeks will be a bit slower than usual, because I'll be riding during the day, and only able to post at night. Apologies in advance if support feels less "spectacularly fast" during these two weeks: I'll do my best to respond as quickly as I can.

Thanks, as always, for using SuperDuper: we couldn't do any of this without you, and we'll continue to do our best to make SuperDuper an application worthy of your praise and personal recommendations.

Spinning round and round Saturday, August 07, 2010

The What

Ever been confronted by a Spinning Pinwheel of Death on launch of SuperDuper! on Snow Leopard and, while waiting for it to stop doing that, raised your hands to the sky and cursed me and the ground I walk on?

Hey, me too (without the self-cursing—kids, remember, if you curse yourself you'll grow hair on your soul)! But (and here's the important part) it's not unique to SuperDuper!. You'll also see this when you do a File > Open, File > Save As, and many other actions. It's not just a delay due to drive spin-up, and there's a fix!

The Why

I can't pretend to totally understand why this happens, so please don't ask, but it seems that the dyld cache (if you don't know what that is, read it as "stuff deep inside OS X") gets in some sort of weird state, and that state is often seen, up in userland, as these annoying, inexplicable SPOD delays.

Might be best to think "damnable munchkins" is the real reason why. Or circuit ants. Or blame the iPhone 4's antenna: it's Mark Papermaster: he's taking the blame for everything else and, at this point, probably wouldn't mind.

The First Fix (for Users Uncomfortable with Terminal)

The easiest way to fix this is to power off your Mac, hold down the Shift key, keep it down and power back on. Keep Shift down all the way to the desktop. This starts your Mac, more slowly, in Safe Boot, which automatically rebuilds this cache (among other things).

When you're totally logged in and running, try running SuperDuper! or your other app that was slow with File > Open. You'll note that it's quite fast.

Now, restart your Mac normally. And—check that out—it's still fast! Yay!

The Second Fix (for the Terminal Savvy User, or the Bold and Brave)

If you're comfortable with Terminal, you can do this without the Safe Boot restart:

  1. Open Terminal.
  2. Copy the following to the clipboard:

    sudo update_dyld_shared_cache -force

  3. Paste that into Terminal.

  4. Press Return at the end.
  5. When prompted, enter your password (it'll be invisible) and press Return.
  6. When it's completed and the prompt returns, quit Terminal and restart your Mac.

Poof! Problem solved.

In Sum

Hopefully this worked for you, and was useful. Enjoy your faster Mac, and stop cursing me!

SuperDuper! v2.6.2 now available! Tuesday, October 13, 2009

It's been in test for quite a while, and we're happy to say that SuperDuper! v2.6.2 has been released!

I understated the speed improvements we've measured in the press release and collateral, but we really are seeing 3x improvements in-house, which is pretty great. We'll see how it works for you.

Anyway: I hope you enjoy the new version as much as we enjoy being able to provide it to you.

How Ya Doin’? Tuesday, September 22, 2009

Just a quick post to let you know how things are going with SuperDuper! and the upcoming v2.6.2 (no release date yet, so don't ask—hence this post). So far, we've:

  • Implemented a fix for the problem where we don't preserve compression under Snow Leopard
  • Ensured that extra data is no longer copied with back-to-back Smart Updates (as expected, a side-effect of the compression issue)
  • Resolved the various other issues mentioned in the last post (although we haven't come up with a fix for the scheduling/settings issue yet)

These fixes have been in external testing for over a week, and the feedback has been excellent so far.

Since we hate releasing updates that have nothing but "fixes" in them, in addition to the above, we've made some changes that significantly speed up the speed of copying, with both Smart Updates and full copies.

In our internal tests, confirmed by our external testers, copying under Leopard and Snow Leopard (with larger data sets) is up to (wait for it) three times faster, especially with fast destination devices.

Needless to say, both of these changes involved "cracking open the cloner", which requires significantly more testing than a tweak to the UI or some "simple" changes to higher level aspects of the copying process, but progress is very good and we're looking forward to getting this into your hands as soon as we (and our external testers) are satisfied with it.

Note that I'll be at C4 this weekend, so support might be a bit slower than usual from Friday until Monday. If you're at C4 too, say Hi!

Squashed Friday, September 04, 2009

Well, a week has gone by, and 2.6.1 has been downloaded and run by an awful lot of you. It seems to be working well, too—which isn't to say it's perfect so, once again, no unicorn.

Here's what we're working on right now.

Individual file compression under Snow Leopard is not maintained on the backup

This kind of caught us by surprise, frankly. When we checked our copy results, we found that the sizes of files were the same on source and destination, and assumed the files were, indeed, being copied with compression.

Nothing like an assumption to make one feel like an ass. Or an umption. Whatever.

Anyway, what's happening is the OS is hiding the compression from us... and we didn't realize it. For the nerdly among us, here's some terminal commands and output that you'll find interesting:

$ ls -l /Applications/iCal.app/Contents/MacOS/
-rwxr-xr-x  1 root  wheel  9123088 Jul 16 03:42 iCal
$ cp /Applications/iCal.app/Contents/MacOS/iCal ~/iCal
$ ls -la ~/iCal 
-rwxr-xr-x  1 taiko  taiko  9123088 Sep  4 17:16 /Users/taiko/iCal

OK! So, I copied a file, and as you can see, the size of the file is the same when I started and ended: 9123088. This is what you'd see with SuperDuper!, too (except with the ownership and stuff preserved). So, everything's cool, right?

Well, no. At a lower, block level:

$ du /Applications/iCal.app/Contents/MacOS/iCal 
6776 /Applications/iCal.app/Contents/MacOS/iCal
$ du ~/iCal 
17824 /Users/taiko/iCal

Well, look at that: iCal, despite being copied with a system tool, is now nearly 3x the size in blocks after being copied, but the same size at a high level.

Great.

Needless to say, that's why we got confused. The sizes were the same. But they weren't... and you could only tell by looking at the amount of space the files took on the drive, not the file's size.

Note that cp, tar, Finder, etc all behave the same way with this: the file ends up getting expanded. The only one that acts differently, when given special parameters, is ditto, and that's likely because it's used by the installer (which puts these files on the drive in the first place).

We've got a good handle on this, and we're in the process of modifying SuperDuper! to retain the compression in the next update.

Some files get recopied every backup

We're aware that some files (specifically, files in /Applications, /System and /Library) are being recopied each time you back up, which significantly slows down repeated Smart Updates.

Our initial investigation has shown that while we thought this was related to prebinding, this is actually related to the compression issue, above. Specifically, compressed files, when they expand during copying, compare differently with flags and with certain time fields.

We didn't catch this because our confirmation tests didn't use compressed files. We did know that we were recopying files, but due to the fact that the end result was fine, and Snow Leopard shipped four weeks earlier than we expected, we just didn't have time to investigate the problem fully, especially since fixing it would involve the most critical and most time consuming to test part of SuperDuper!—something that we'd frozen many weeks before.

In other words, considering the issue was basically one of performance, we thought it was better to get the update out day-and-date with this known issue, than hold it back for the week or so it would take to investigate and fix (and then however long it took to test, based on what we had to change).

This should also be fixed in the next update.

Slower performance

We've had some reports of slower performance in the new version, which is more than a little weird, since the base-level copy calls are the same. Part of that is due to the re-copying, above, but we're looking into it further to see if there has been a regression, and if so we'll fix it as soon as we confirm and find the cause.

People aren't recreating their schedules

This version of SuperDuper! requires that people recreate their schedules due to some problems reading in parts of the old settings files. Many users aren't doing this, so we're trying to improve our error handling to detect and force the schedule to be recreated, rather than just asking people to do so.

UTF8 encoded URLs in AppleScript bite me again

So I'm an idiot. I can't tell you how many times I've attacked this specific problem in AppleScript, but I blew it again in the schedule driver in some cases, so drive names with some unusual characters (accents, etc) won't always run when scheduled.

Again.

I've already fixed this problem and it should be in the next update. It's been tested with both regular Unicode characters (e.g. ) and composed sequences (e.g. é). Note, though, that I haven't managed to get AppleScript to work with a drive named using the Alpha character (α) with backup-on-mount (reported by Glenn—thanks), because Finder fails when I try to use that character in a built-up path. Anyone who's awesome with this stuff in AppleScript please contact me if you have any ideas, because I'm fresh out.

Japanese users and backslashes

Back in the days of Panther, there was a weird bug where backslashes () didn't work in AppleScripts, because they were confused with the Yen symbol when you were in Japanese language mode.

That was supposedly fixed in the compiler back then... but in Snow Leopard, the problem seems to be back (it works properly in the Script Editor but not in the command-line compiler osascript), and I used some backslashes in the updated schedule driver. That's been fixed as well—thanks to Tohru for reporting the problem.

I think that just about covers the issues we're working on right now at a high priority... back to it.

SuperDuper! v2.6.1 released Saturday, August 29, 2009

Rather than taking the time to write a mildly amusing "we suck" press release that announces this day-after-the-big-release update, let me just say it fixes the stuff I mentioned in the last post, followed by ringing a dinner bell.

Come and get it!

Page 4 of 13 pages « First  <  2 3 4 5 6 >  Last »