Shirt Pocket Discussions  
    Home netTunes launchTunes SuperDuper! Buy Now Support Discussions About Shirt Pocket    

Go Back   Shirt Pocket Discussions > SuperDuper! > General

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 03-13-2010, 05:50 PM
diamondsw diamondsw is offline
Registered User
 
Join Date: Oct 2009
Posts: 16
SuperDuper Scheduling/cron failures

First off, a scheduled copy that I previously deleted is still attempting to run. Nightly, I get messages in my UNIX mailbox (nothing I ever would have noticed, except for a stray line in the terminal one day) complaining that cron can't find a file, and points to my old saved settings that had been scheduled. I've searched my entire system for cron files that might be related to this and found nothing.

Suggestions:
  • First of all, I'd say don't use cron. Anything it can do launchd can do better, and the files are in standard user-visible locations and more readable.
  • If you're still using cron, please check for stray entries such as the one that I'm seeing. I'd delete it myself except I can't find the thing.

In researching this, I was a bit appalled at the potential for breakage in the "backup-on-mount" system.
  • Two different launchd scripts watch for either login or /Volumes events
  • They launch an alias in the home directory that's to a script located inside the SuperDuper application bundle
  • The script does some diskinfo magic and jury-rigs some plists for Tiger users, seemingly to get a list of what disks are attached.
  • Script then iterates through all scheduled copies in the user's home folder looking for ones that run on mount, and if they contain the newly mounted drive name. What if I have two backup-on-mount destinations called "Backup" and "Backup 2"?
  • If found, environment variables are set and a new application (which appears to be inside the scheduled copy package) is launched which will launch SuperDuper and the scheduled copy. Not sure why an app is needed for this; just script it since you're already in AppleScript.
It requires two launchd scripts, an alias to resolve, is dependent on the workings of a command-line program which has changed format in the past, assumes disk layout is in a standard state (e.g. home directory hasn't moved), then is dependent on the internal XML of the scheduled copies, assumes disk names won't contain other disk names, sets an environment variable that only this one app knows, and fires the app.

Code:
launchd (2) -> alias -> AppleScript -> diskinfo (10.4 vs 10.5) -> iterate through folder (hope it didn't move, parse XML) -> set environment variable -> open a buried ".app" -> open SuperDuper.app
Suggestion:
  • Use a single launchd daemon that runs on either login or on /Volumes changing.
  • Use a single AppleScript or compiled daemon in a known location (/usr/local/bin is traditional), which at the end calls SuperDuper directly with the appropriate parameters.

My attempts at replacing this system have been difficult, as SuperDuper's "error" keyword in AppleScript conflicts with the language itself and can't be used:

Quote:
status (idle/running/done/error, r/o) : Returns SuperDuper's current status
"Error" is reserved by the AppleScript language:

Quote:
error
Signals an error in a script.
Syntax
error [ errorMessage ] [ number errorNumber ] ¬
***[ partial resultresultList ] ¬
***[ from offendingObject ] [ to expectedType ]
I get as far as checking volume mounts, ensuring the necessary volumes are there, put up a brief message, and get all the way into telling SuperDuper to copy - and then get stuck on the error status. So instead of checking for an error you have to check all other statuses and from that figure out there was an error. Kludgy.

Suggestion:
  • Use something not reserved by AppleScript.
Reply With Quote
  #2  
Old 03-13-2010, 07:39 PM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,354
Send a message via AIM to dnanian
Needless to say, SuperDuper!'s scheduling facility is older than launchd, and there's little reason to move from cron to launchd: it doesn't do this kind of thing "better", it's just "newer". (In fact, cron's been around for a very long time, and is well known and tested.)

If you want to edit the crontab, you can use "crontab -e" and remove the bad entry... but I don't know how you ended up with a 'stray'.

As far as "Backup on Mount" goes, I'm well aware of how it works, and I think you'll find it's less fragile than you imply. To make things easy to remove, all executables (etc) are kept in user accessible areas - /usr/local/bin, while traditional, is not removed when the application is removed (and the alias is updated to point at the right place each launch).

Two destination called "Backup" and "Backup 2" aren't a problem, because the actual full volume name is checked once a quick pass is made to subset the settings being tested. And we don't duplicate the functionality in the schedule driver because, well, why maintain two executables that do the same thing?

Also, we can't use one launchd daemon: it doesn't work properly in all versions of OSX.

Alas, I know about the "error" thing - it's a known, logged issue (you're the first person other than me to notice, so kudos).
__________________
--Dave Nanian
Reply With Quote
  #3  
Old 03-14-2010, 06:36 AM
diamondsw diamondsw is offline
Registered User
 
Join Date: Oct 2009
Posts: 16
Quote:
Originally Posted by dnanian View Post
Needless to say, SuperDuper!'s scheduling facility is older than launchd, and there's little reason to move from cron to launchd: it doesn't do this kind of thing "better", it's just "newer". (In fact, cron's been around for a very long time, and is well known and tested.)
It's also deprecated and could be removed entirely any time. Launchd is much more flexible, and in my opinion (but just an opinion) more readable and readily accessible than cron. Cron's advantage is it's familiar... to people who know cron. I thought I knew cron, but vixie-cron is quite a bit different from the traditional editing of /etc/crontab.

I'd have also thought that with the backup-on-mount system using launchd and Applescript, it would make sense for the scheduling system to also use the same framework. What exactly does cron allow SuperDuper to do that launchd does not?

Quote:
Originally Posted by dnanian View Post
If you want to edit the crontab, you can use "crontab -e" and remove the bad entry... but I don't know how you ended up with a 'stray'.
SuperDuper is the only thing accessing it. So, I don't know what to tell you.

Quote:
Originally Posted by dnanian View Post
As far as "Backup on Mount" goes, I'm well aware of how it works, and I think you'll find it's less fragile than you imply. To make things easy to remove, all executables (etc) are kept in user accessible areas - /usr/local/bin, while traditional, is not removed when the application is removed (and the alias is updated to point at the right place each launch).
I'd hope you know how it works! :-) Still, I can't help but think that there's got to be a fair amount of code in there devoted to keeping the various parts of that working and checking for errors that wouldn't necessarily be required with a simpler setup. It's an awful lot of glue to accomplish:
  1. Check for a new mount.
  2. Check if a backup script references it.
  3. Run backup script(s).

I suppose when you get down to it, the things I don't understand are the need for two launchd agents instead of one, and the opaque ".app" that starts things rather than just scripting SuperDuper directly. The alias I'm more comfortable with, as I can see that gives a stable path for the launchd agent and alleviates the concerns about moving SuperDuper.

Quote:
Originally Posted by dnanian View Post
Two destination called "Backup" and "Backup 2" aren't a problem, because the actual full volume name is checked once a quick pass is made to subset the settings being tested.
Good to know, thanks.

Quote:
Originally Posted by dnanian View Post
And we don't duplicate the functionality in the schedule driver because, well, why maintain two executables that do the same thing? Also, we can't use one launchd daemon: it doesn't work properly in all versions of OSX.
The devil's advocate in me says, then why maintain the multitude of pieces that currently backup on mount and schedule copies? There are so many bits of Applescript, stub applications, and command line utilities, whereas I'd think Launchd could handle the events, and at most a single daemon or script could handle passing control to the main app. I know it's grown over time - especially working around OS differences - but it still just seems overwrought.

I suppose when you get down to it, there's no arguing with results - the only issue I have is this phantom cron entry; backups run just fine - and your user base would likely agree that all is well. Something just feels off, and when I see that many pieces, I naturally worry about breakage. The more complex any system is, the more likely it is that something can fail. Of course, given your intimate work with filesystems, you know about that with far, far more experience than I - I've read the travails on your blog.

Quote:
Originally Posted by dnanian View Post
Alas, I know about the "error" thing - it's a known, logged issue (you're the first person other than me to notice, so kudos).
Heh, pretty sure I reported this one back in the day (2.0 era) when I first created my own scheduling script. I'd still use it, but my backup routine is now much more based on Time Machine for continuous backup, and backup-on-mount for occasional bulk backups to offline storage.

Last edited by diamondsw; 03-14-2010 at 06:40 AM.
Reply With Quote
  #4  
Old 03-14-2010, 11:58 AM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,354
Send a message via AIM to dnanian
Cron is not deprecated (see "man cron" - no menton of deprecation): in fact, it's implemented with launchd. Launchd doesn't offer any more flexibility for timer-scheduled events (I had hoped it would, but it doesn't). Rewriting scheduling to use launchd wouldn't really generate any tangible benefits, other than to use "the new shiny". There's just no compelling argument to do so at present.

As far as vi goes, well, navigate to the line you want to remove with the cursor keys, press 'd' twice to delete the line, then ":wq" to write and quit vi. Alternatively, if you uncheck the time part of all your schedules and then use "crontab -r", it'll remove the tab. It'll then get recreated when you turn the schedules on again.

The two daemons - that is, the login one and the 'on mount' one - do entirely different things. One sets up the backup-on-mount environment by initializing the list of volumes, which handles situations like logging in and out of an account where temporary items aren't removed. One actually responds to mount events. They're quite different, even though they call the same program (with different parameters) to do the actual work.

I just don't agree that this stuff is overly complex. Each part is there for a reason... there's an 'automatic run' driver that's used to run a given set of settings for an event (either scheduled, on mount, on button press or any other event), there's a system tool for running on schedule that we leverage (and was there well before launchd, and offers the same functionality with the same limitations), and there's launchd for triggering running on mount.

Each group of settings has its own 'automatic run' driver because they're designed to be modifiable by advanced users on a 'per setting' basis. You need a separate backup-on-mount 'driver' because you don't know where the 'automatic run' driver is when the event fires (and there can be many), nor what settings should be used, and that takes an event ("something mounted") and that determines how to respond to the event, and with what.

Yes, we could put all that not-really-that-related-but-working-together-to-accomplish-a-task logic in one giant centrally-located tool, and eliminate the user-modifiable part, but that gets rid of useful functionality (to a small number of users, of course), and doesn't really 'fix' any problem (and, it couples things closer than I would want). I mean, it'd be like taking every command-line tool in Unix that deals with 'files' and combining them into one huge tool with a zillion parameters to 'streamline maintenance'. They tried that in CPM: it was called 'pip', and it was a huge mistake.

(As an aside, this stuff is done in AppleScript, rather than in Objective-C, to show how you can use AppleScript to accomplish system-level things in OS X. Personally, AppleScript drives me crazy—I can't tell you how much I struggled with it when I wrote these two parts—but it's more understandable than the various shell script languages to most users, and was the right tool for the job.)

Anyway, things sometimes feel 'off' when you haven't gone through the process and history of creation. Does that mean everything in SuperDuper! is perfect? No - but there are usually good reasons to do what we've done...
__________________
--Dave Nanian
Reply With Quote
  #5  
Old 03-14-2010, 06:27 PM
diamondsw diamondsw is offline
Registered User
 
Join Date: Oct 2009
Posts: 16
All good info, and I appreciate the time to educate. Bringing up the UNIX metaphor was a good one, as definitely crystalizes the "many specific parts" approach. I suppose I just wasn't in that mindset due to the presence of aliases, AppleScript, and ".app".

Saying cron is based on launchd is a bit of a stretch. The cron process itself is launched by launchd (since we don't have an "init" anymore), but from then on all cron tasks are handled by cron itself and launchd has nothing to do with them. That's just due to launchd replacing init.

cron was deprecated in 10.4 (four and a half years ago), whether explicitly or implicitly. This "unique" way of scheduling certainly took me forever to find - the only reason I did was when I got error messages as a result of it. Since you require 10.4 or later for SuperDuper, you always have launchd available. From an end-user perspective I can see exactly where launchd is configured (Library/Launch(Agents|Daemons)), monitor, start, and stop processes as needed (launchctl), and make changes easily with a multitude of tools (Lingon, etc). For cron there's no reason to suspect it's running at all since Mac OS X doesn't use it, there are no configuration files on disk that I've been able to find, and the default editor (vi - which I finally managed to change to nano) is about as arcane and unfriendly as it gets.

Unless there's a specific launchd bug on 10.4.11 that I don't know of, I don't see it buying you anything. The only change I see would be writing a plist to /Library/LaunchAgents/ instead of a line to cron.

As for AppleScript vs Objective-C, I can completely empathize. In their attempt to make AppleScript "readable", they certainly made the syntax unpredictable. One of those things that sounds good in theory, but doesn't actually work all that well. However, AppleScript is the only option for most people to automate Mac OS X applications, so it is what it is. Any chance of that "error" keyword getting a tweak?
Reply With Quote
  #6  
Old 03-14-2010, 06:38 PM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,354
Send a message via AIM to dnanian
Having a new facility that does the same kind of thing cron does doesn't deprecate cron. As I said, there is nothing in cron's man pages that indicates cron is deprecated, and there's no such thing as 'implicit deprecation': something is either deprecated or it isn't. And, in this case, it isn't.

Rewriting things to use launchd rather than cron really doesn't make much sense unless we're bored and have nothing better to do. Doesn't mean it wouldn't be nice, just that it's not necessary (and most end users don't know anything about launchd either; it's not like it's less obscure than cron is).

For all AppleScript's flaws, it is also quite powerful. You can get a lot of done in a few statements... once you figure out what those statements need to be.

The error thing has been on the list for a while... it's just not high priority enough to rise to the top yet...
__________________
--Dave Nanian
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Failed to enable ownership on SuperDuper BackUp Anne West General 1 02-03-2010 02:38 PM
Superduper fails after upgrade to Snow Leopard josephc General 2 09-12-2009 11:18 PM
Needed: Time Machine and SuperDuper guidance kapalama General 7 08-13-2009 08:55 AM
New to Mac and to SuperDuper - Need a little help MikeG General 15 02-14-2008 11:35 PM
A word of praise for SuperDuper! MMM General 3 06-21-2006 11:08 PM


All times are GMT -4. The time now is 03:07 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.