launchTunes

SuperDuper v3.2.5 Now Available! Tuesday, June 25, 2019

Watch me pull a rabbit out of this hat.

You'd think, given I've written everything here, I'd learn to pay attention to what I've said in the past.

Let me explain. In v3.2.4, we started getting occasional reports of scheduled copies running more than once. Which seemed kind of crazy, because, well, our little time-based backup agent—sdbackupbytime—only runs once a minute, and I'd reviewed the code so many times. There was no chance of a logic error: it was only going to run these puppies one time.

But it was, sometimes, regardless. And one of the things I had to change to complete Smart Wake was to loop to stay active when there was a backup coming in the next minute. But the code was definitely right. Could that have caused this to happen?

(Spoiler alert after many code reviews, hand simulations and debugger step-throughs: NO.)

So if that code was right, why the heck were some users getting occasional multiple runs? And why wasn't it happening to us here at Shirt Pocket HQ?

Of course, anyone who reported any of these problems received an intermediate build that fixed them all. We didn't rush out a new update because the side effect was more copies rather than fewer.

Secondary Effects

Well, one thing we did in 3.2.4 was change what SuperDuper does when it starts.

Previously, in order to work around problems with Clean My Mac (which, for some reason, incorrectly disables our scheduling when a cleanup pass is run, much to my frustration), we changed SuperDuper to reload our LaunchAgents when it starts, in order to self-repair.

This worked fine, except when sdbackupbytime was waiting a minute to run another backup, or when sdbackuponmount was processing multiple drives. In that case, it could potentially kill the process waiting to complete its operation. So, in v3.2.4, we check to see if the version is different, and only perform the reload if the agent was updated.

The known problem with this is that it wouldn't fix Clean My Mac's disabling until the next update. But it also had a secondary effect: these occasional multiple runs. But why?

Well, believe it or not, it's because of ThrottleInterval. Again.

Readers may recall that launchd's ThrottleInterval doesn't really control how often a job might launch, "throttling" it to only once every n seconds. It actually forces a task to relaunch if it doesn't run for at least n seconds.

It does this even if a task succeeds, exits normally, and is set up to run, say, every minute.

sdbackupbytime is a pretty efficient little program, and takes but a fraction of a second to do what it needs to do. When it's done, it exits normally. But if it took less than 10 seconds to run, the system (sometimes) starts it again...and since it's in the same minute, it would run the same schedule a second time.

The previous behavior of SuperDuper masked this operation, because when it launched it killed the agent that had been re-launched: a secondary, unexpected effect. So the problem didn't show up until we stopped doing that.

And I didn't expect ThrottleInterval to apply to time-based agents, because you can set things to run down to every second, so why would it re-launch an agent that was going to have itself launched every minute? (It's not that I can't come up with reasons, it's just that those reasons are covered by other keys, like KeepAlive.)

Anyway, I changed sdbackupbytime to pointlessly sleep up to ThrottleInterval seconds if it was going to exit "too quickly", and the problem was solved...by doing something dumb.

Hey, you do what you have to do, you know? (And sometimes what you have to do is pay attention to your own damn posts.)

Queue'd

Another big thing we did was rework our AppleScript "queue" management to improve it. Which we did.

But we also broke something.

Basically, the queue maintains a semaphore that only allows one client at a time through to run copies using AppleScript. And we remove that client from the queue when its process exits.

The other thing we do is postpone quit when there are clients in the queue, to make sure it doesn't quit out from under someone who's waiting to take control.

In 3.2.4, we started getting reports that the "Shutdown on successful completion" command wasn't working, because SuperDuper wasn't quitting.

Basically, the process sending the Quit command was queued as a task trying to control us, and it never quit...so we deferred the quit until the queue drained, which never happened.

We fixed this and a few similar cases (people using keyboard managers or other things).

Leaving Things Out

As you might know, APFS supports "sparse files" which are, basically, files where "unwritten" data takes up no space on a drive. So, you might have a file that was preallocated with a size of 200GB, but if you only wrote to the first 1MB, it would only take up 1MB on the drive.

These types of files aren't used a lot, but they are used by Docker and VirtualBox, and we noticed that Docker and VirtualBox files were taking much longer to copy than we were comfortable with.

Our sparse file handling tried to copy a sparse file in a way that ensured it was taking a minimal amount of space. That meant we scanned every read block for zeros, and didn't write the sections of files that were 0 by seeking ahead from that point, and writing the non-zero part of the block.

The problem with this is that it takes a lot of time to scan every block. On top of that, there were some unusual situations where the OS would do...weird things...with certain types of seek operations, especially at the end of a file.

So, in 3.2.5, we've changed this so that rather than write a file "optimally", maximizing the number of holes, we write it "accurately". That is, we exactly replicate the sparse structure on the source. This speeds things up tremendously. For example, with a typical sparse docker image, the OS's low-level copyfile function takes 13 minutes to copy with full fidelity, rsync takes 3 minutes and doesn't provide full fidelity, whereas SuperDuper 3.2.5 takes 53 seconds and exactly replicates the source.

That's a win.

Don't Go Away Mad...

In Mojave 10.14.4 or so, we starting getting reports of an error unmounting the snapshot, after which the copy would fail.

I researched the situation, and in this unusual case, we'd ask the OS to eject the volume, it would say it wasn't able to, then we'd ask again (six times), and we'd get an error each time...because it was already unmounted.

So, it would fail to unmount something that it would then unmount. (Winning!)

That's been worked around in this update. (Actual winning!)

Improving the Terrible

As you've seen, Apple's process for granting Full Disk Access is awful. There's almost no guidance in the window—it's like they made it intentionally terrible (I think they did, to discourage people from doing it).

We'd hoped that they'd improve it, and while some small improvements have been made, it hasn't been enough. So, thanks to some work done and generously shared by Media Atelier, we now provide instructions and a draggable application proxy, overlaid on the original window. It's much, much better now.

Thanks again and a tip of the pocket to Pierre Bernard of Houdah Software and Stefan Fuerst of Media Atelier!

eSellerate Shutdown

Our long-time e-commerce provider, eSellerate, is shutting down as of 6/30. So, we had to move to a new "store".

After a long investigation, we've decided to move to Paddle, which—in our opinion—provides the best user experience of all the ones we tried.

Our new purchase process allows payment through more methods, including Apple Pay, and is simpler than before. And we've implemented a special URL scheme so your registration details can now be entered into SuperDuper from the receipt with a single click, which is cool, convenient, and helps to ensure they're correct.

Accomplishing this required quite a bit of additional engineering, including moving to a new serial number system, since eSellerate's was going away. We investigated a number of the existing solutions, and really didn't want to have the gigantically long keys they generated. So we developed our own.

We hope you like the new purchase process: thanks to the folks at Rogue Amoeba, Red Sweater Software, Bare Bones Software and Stand Alone—not to mention Thru-Hiker—for advice, examples and testing.

Note that this means versions of SuperDuper! older than 3.2.5 will not work with the new serial numbers (old serial numbers still work with the new SuperDuper). If you have a new serial number and you need to use it with an old version, please contact support.

(Note that this also means netTunes and launchTunes are no longer available for purchase. They'll be missed, by me at least.)

Various and Sundry

I also spent some time improving Smart Delete in this version; it now looks for files that have shrunk as candidates for pre-moval, and if it can't find any space, but we're writing to an APFS volume, I proactively thin any snapshots to free up space on the container.

All that means even fewer out of space errors. Hooray!

We also significantly improved our animations (which got weird during 10.13) by moving our custom animation code to Core Animation (wrapping ourselves in 2007's warm embrace) and fixed our most longstanding-but-surprisingly-hard-to-fix "stupid" bug: you can now tab between the fields in the Register... window. So, if you had long odds on that, contact your bookie: it's going to pay off big.

With a Bow

So there you go - the update is now available in-app, and of course has been released on the Shirt Pocket site. Download away!

Ocean of words Thursday, May 03, 2007

For those who haven't heard me blather at length about all things SuperDuper! and Shirt Pocket, there's a 45 minute interview of me up at MacVoices.

Remember: the game rules require you to drink every time you hear me say "Um", "Uh" or audibly gesticulate. Do not play while driving.

New netTunes and launchTunes! Wednesday, August 23, 2006

Shirt Pocket releases netTunes 2.3 and launchTunes 1.1
The Macworld Eddy Award Winning “No Compromises” iTunes Remote - now Universal!

Weston, MA – August 23, 2006:  Shirt Pocket announces the immediate availability of netTunes 2.3, the latest update to the Macworld Eddy Award winning remote control for iTunes, and the perfect companion to Apple’s AirPort Express and launchTunes 1.1, the application that guarantees your shared iTunes libraries are available without all that pesky walking.

netTunes lets you control iTunes running on one Macintosh from another, using iTunes’ native interface. You get the same window, the same playlists, the same capabilities. You simply run netTunes and take complete control of the “remote” iTunes from any Macintosh in your house. It’s that easy — as easy as iTunes itself!

“It’s been a long time coming, but the new versions of netTunes and launchTunes are ready to go” says David Nanian, the founder of Shirt Pocket. “But we didn’t just recompile—we’ve taken advantage of both the Intel and Power PC platforms by improving performance across the board, and polished the user experience as well.”

netTunes and launchTunes are available for immediate download at the Shirt Pocket web site http://www.shirt-pocket.com. Users can evaluate all of the capabilities of netTunes for free for 30 minutes at a time; full licenses cost $19.95, and can be ordered at the Shirt Pocket web site, or directly from the application. launchTunes costs $7, and the Tune Suite—a bundle of netTunes and launchTunes—is offered at $23.95 - a $3 savings.

About Shirt Pocket
Shirt Pocket, based in Weston, Massachusetts, was formed in late 2000 as a Macintosh-only shareware creator and publisher. Shirt Pocket’s first product, the Eddy award winning netTunes, lets users control iTunes on one Mac from any other Mac on the network with iTunes own intuitive user interface. launchTunes, Shirt Pocket’s second product, made iTunes’ playlist sharing practical by automatically launching iTunes on remote servers when needed. And its third, the Eddy award winning SuperDuper!, is one of the most highly acclaimed backup/cloning programs available for the Mac. All are available from the Shirt Pocket web site at http://www.shirt-pocket.com.

Shirt Pocket was started by David Nanian, co-founder of UnderWare, Inc, and one of the original authors of the BRIEF programmer’s editor and Track Record bug tracking system.

Taiko Monday, August 21, 2006

A few quick pictures of Taiko as I prepare for the rollout of the new netTunes and launchTunes this week…

Young Taiko
Taiko Sleeping
Taiko on Rug - 11 Weeks

Waiting… Sunday, July 30, 2006

OK! The testing of netTunes and launchTunes, its purchasing process, Universal Support and integrated store is complete. Things are basically ready to go, except for one thing—I’m waiting for a new version of the eSellerate Integrated Web Store library to be released.

This was supposed to go sometime a week or two ago, but unfortunately got delayed. It’s important, though, because the store installation procedure leaves a file in /tmp that, for some reason, Tiger feels is necessary to move into “Recovered Items” in your trash.

They’ve been working on a fix for that, and I’ve been waiting for that fix to become available. Any day now, I’m told, and I’m passing that savings on to you!

What’s taking so long? Thursday, July 06, 2006

It’s been difficult, these last few months, to find the time necessary to work on netTunes and launchTunes while working on/supporting SuperDuper! and navigating the waters of decline with Ketzl.

netTunes was pretty much done back in April, save for some online store issues—I want to support PayPal, and that means moving to the new “Embedded Web Store” (which is also required for Universal apps). Doing that involves doing some new graphics/layout for the custom pages.

launchTunes, though, was harder—ironic, since it’s a much simpler program. The big problem was that launchTunes hadn’t yet been converted to a drag-and-drop install, and I’d done some work extending its capabilities that hadn’t been completed.

Plus, I’d learned more about Cocoa in the few years since its original release (it’s never been updated)… so, it was time to throw out the old and rewrite the whole thing. Which, given the time constraints, took longer in calendar time than anticipated.

I’m happy to say that I managed to finish up the main launchTunes application, with its auto-install capabilities, while flying back from Chicago the other day. Everything tested out great, so now I’m working on the two apps’ graphics and online store stuff.

Once that’s done, the Universal versions of netTunes and launchTunes will finally get into your hands. Sorry it’s taken so long!

Hoarse-ing around Thursday, May 04, 2006

Despite being down and mostly out with a cold/flu, I’ll be appearing on The Tech Night Owl LIVE tonight, May 4th, talking about SuperDuper!, netTunes, launchTunes, and desperately trying to not cough up a chunk of lung.

You can tune into the broadcast from 6:00 to 8:00 PM Pacific, 9:00 to 11:00 PM Eastern, at http://www.techbroadcasting.com. An archive of the show will be available for downloading and listening within four hours after the original broadcast.

You can also access the show’s Podcast feed, available at: http://www.techbroadcasting.com/nightowl.xml.

Going Universal Wednesday, February 01, 2006

Our “production” iMac hit the Shirt Pocket loading dock (aka my front step) yesterday, and I’ve been busily working on updates to netTunes and launchTunes while Bruce works on completing the next update to SuperDuper!, which looks like it will be Universal as well.†

It’s been nice to get back to Xcode after some time off (my design/marketing/documentation/support/etc duties keep me away from it more than I’d like), and it didn’t take more than a few hours to do the netTunes conversion—it’s up and running as I type, and works great.

The big challenge with netTunes was the screen “scraping”—the code has some byte-order dependencies because of pixels, and I wasn’t looking forward to re-writing the “inner loop” to handle all the various cases with Thousands and Millions of colors using both MMX and Altivec. Blegh.

Fortunately, there was an elegant solution. Since the Intel Mac is Tiger-only, I was able to make use of some the new Accelerate framework APIs to eliminate byte order dependencies and—at least under Tiger—about two pages of nasty code. Since the Intel Macs can’t run Panther, I didn’t have to change the already-working parts. The result? Both the Intel and PPC versions benefit, at least under 10.4.

It’s always a good feeling when you can delete code and leave the hardware-specific optimization up to the guys who live and breathe vector processing units, let me tell you.

Anyway, we’re working hard to get the Universal binaries out the door. Obviously, it’s important to do extensive testing using real, production hardware—and we’re doing just that.

† - Updated for the exclamation point haters. You know who you are.

Page 1 of 1 pages