Shirt Pocket Discussions  
    Home netTunes launchTunes SuperDuper! Buy Now Support Discussions About Shirt Pocket    

Go Back   Shirt Pocket Discussions > SuperDuper! > General

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 01-16-2006, 04:31 PM
Syzygies Syzygies is offline
Registered User
 
Join Date: Jan 2006
Posts: 23
A strategy for managing SD! jobs via AppleScript

My strategy for managing SD! jobs via AppleScript is this:

First, schedule each desired SD! job from within SD!, with any schedule that won't execute in the near future. This creates a settings package in the folder Library/Application Support/SuperDuper!/Scheduled Copies/, with a name such as Smart Update aOSX from OSX.sdsp. Move these settings packages to your own folder; this allows them to run independently of SD!'s built-in scheduling feature, as I noticed by reading their AppleScript code. Now go back in SD! and delete these scheduled jobs. The settings packages will survive intact; for the moment, SD! has no idea where they went.

Each of these settings packages contains a compiled AppleScript called Copy Job. One can Show Package Contents and then manually launch Copy Job, to run the corresponding SD! job. The log files for these jobs are also stored in these settings packages.

Alternatively, one can call these Copy Job scripts from within AppleScript. The calling script can then be run manually, or scheduled using cron or a GUI for cron such as CronniX. I describe below how I resolved the issues that arose for me in writing and testing such a script.

For a variety of reasons, I don't want my backup hard drives mounted when I'm not backing up to them. I also rotate hard drives as backup media, and I want a script to gracefully notice when media is unavailable, and move on. So I mount each volume as needed from within AppleScript, and eject it afterwards.

I found that I needed to eject volumes by telling the Finder, but this has the effect of ejecting all volumes on a given physical hard drive. This affects both one's strategy in partitioning one's drives, and one's code: One needs to avoid timing issues where ejecting the last volume used could also eject the next volume needed.

Along the same lines, all available volumes are mounted whenever a user logs in, so it is necessary to write another AppleScript (not shown, but similar) to eject unwanted volumes on login.

My AppleScript mount routine is written to be as robust as possible for my purposes, rather than to be as concise as possible:

Some code fragments that I found for mounting partitions were vulnerable to volume name issues. The egrep pattern matches an entire line of the diskutil list output, and sed removes the volume name, so awk can find the disk locator in a consistent column. This allows for volume names with embedded spaces, and volume names that contain one another. Still, it is poor form to have two identical volume names available, and this code will behave unpredictably in that event.

I also want to return the status of mount attempts as quickly as possible. mount executes two shell scripts, rather than combining these into one, in order to detect immediately if a volume name isn't even listed in the diskutil list output. Presumably, this is rotating backup media that is intentionally unavailable.

If SD! is open, my script waits for SD! to quit. Similarly, it waits for SD! to quit before proceeding to subsequent copy jobs. This code is based on the source code for the SD! Copy Job scripts.

Each top level copyJob call takes two arguments, the name of the destination volume, and a full AppleScript alias for the SD! Copy Job script to run. AppleScript aliases need to exist at compile time, but they have the advantage of automatically updating themselves when files are moved. The relative unreadability of this argument is a small price to pay for the convenience and feedback one gains.

Links to related threads:

Mounting an external volume before backup

An (Applescript) hack to automate mounting and unmounting external volumes

Here is my complete AppleScript for managing SD! jobs:

Note: I have revised this code at least once. I am preserving the sequence of posted versions in this thread, so that the discussion makes sense. However, I recommend modifying the latest version to your needs, after you understand the code.

Code:
on mount(diskname)
  tell application "Finder"
    if (exists the disk diskname) then return true
    try
      set a to do shell script "diskutil list | egrep '^ +[[:digit:]]+: +[[:graph:]]+ +" & diskname & " +[.[:digit:]]+ GB +disk[[:alnum:]]+$'"
      do shell script "diskutil mount `echo '" & a & "' | sed 's/" & diskname & "//' | awk '{print $5}'`"
    on error
      return false
    end try
    repeat 12 times
      tell application "System Events" to do shell script "sleep 5"
      if (exists the disk diskname) then return true
    end repeat
    return false
  end tell
end mount

on eject(diskname)
  tell application "Finder"
    if not (exists the disk diskname) then return true
    try
      eject disk diskname
    on error
      return false
    end try
    repeat 12 times
      tell application "System Events" to do shell script "sleep 5"
      if not (exists the disk diskname) then return true
    end repeat
    return false
  end tell
end eject

on isSuperDuperRunning()
  tell application "System Events"
    return exists process "SuperDuper!"
  end tell
end isSuperDuperRunning

on waitForSuperDuper()
  repeat while isSuperDuperRunning
    tell application "System Events" to do shell script "sleep 5"
  end repeat
end waitForSuperDuper

on copyJob(diskname, scriptalias)
  if mount(diskname) then
    waitForSuperDuper
    run script alias scriptalias
    waitForSuperDuper
    eject(diskname)
  else
    display dialog "Run Backups: \"" & diskname & "\" is unavailable" giving up after 30
  end if
end copyJob

on run
  copyJob("aOS9", "User:Users:ad:Scripts:Smart Update aOS9 from OS9.sdsp:Copy Job.app:")
  copyJob("aOSX 10.3.9 min", "User:Users:ad:Scripts:Smart Update aOSX 10.3.9 min from OSX 10.3.9 min.sdsp:Copy Job.app:")
  copyJob("aOSX 10.4.4 min", "User:Users:ad:Scripts:Smart Update aOSX 10.4.4 min from OSX 10.4.4 min.sdsp:Copy Job.app:")
  -- copyJob("aUser", "User:Users:ad:Scripts:Smart Update aUser from User.sdsp:Copy Job.app:")
  -- copyJob("aOSX", "User:Users:ad:Scripts:Smart Update aOSX from OSX.sdsp:Copy Job.app:")
end run

Last edited by Syzygies; 01-22-2006 at 10:06 AM. Reason: Please read entire thread for latest code.
Reply With Quote
  #2  
Old 01-16-2006, 07:24 PM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,290
Send a message via AIM to dnanian
Thanks, Syzygies! Hopefully this'll give people even more ideas for what they can do with the SuperDuper! AppleScript interface and built-in Copy Job script...
__________________
--Dave Nanian
Reply With Quote
  #3  
Old 01-22-2006, 10:50 AM
Syzygies Syzygies is offline
Registered User
 
Join Date: Jan 2006
Posts: 23
Revised code

Here is a revised version of my code to selectively mount and unmount volumes, and call SD! scripts unattended. I've sorted out various issues, and it now works fine as a cron job; I rely on it nightly.

Overall, the code has been improved by never telling anything to the Finder. 'unmount' now only unmounts the given volume, without ejecting other volumes on the same disk. It is possible to have an AppleScript such as this that calls 'diskutil' crash on rare occasions. I don't know why, and I'm not sure that I'm actually doing anything wrong. (SD! itself sometimes uses 'disktool' instead of 'diskutil', despite Apple's efforts to wean us from it; perhaps Dave can explain why?)

Also on rare occasions, I'd run a prior version of this script unattended, and one of the volumes failed to unmount. I considered what I would have done manually had I been there, and wrote a wrapper 'unmountVol' to do exactly that. It looks like massive overkill in the calm light of day, but its precautions are entirely harmless for unattended operations.

I leave diagnostic journaling on at all times, as SD! does. If anything odd happens, this is invaluable. Note that all shell script commands are logged, together with any errors returned, even for commands that always return an error. (We wouldn't want to miss it, if one of them returned a different error.)

All forums have a high percentage of lurkers. I google and lurk various places to pick up code fragments to try, and I consider this to be the primary value of posting code on the web; I don't like to use unsupported code that I haven't rewritten myself. To be conservative, if no one else has posted here that this code worked for them, proceed with caution in modifying it for your needs.

This is an AppleScript library file, Backups Library.scpt. Opening it opens the script in the Script Editor:

Code:
property logfile : "/Volumes/User/Users/ad/Scripts/logs/log"

-- initialized, journal, shell, wait : utility routines

property new : true

on initialize()
  if new then
    set logfile to logfile & (do shell script "date '+ %Y-%m-%d.txt'")
    set new to false
  end if
end initialize

on journal(message)
  local wasnew, s, f
  set wasnew to new
  if new then initialize()
  set s to (quoted form of message)
  set f to ">>" & (quoted form of logfile) & "; "
  if wasnew then
    do shell script "echo " & f & "echo " & s & f & "echo " & f
  else
    do shell script "echo `date '+%H:%M:%S'` ' ' " & s & f
  end if
end journal

on shell(command)
  -- use full paths for any commands not in path "/usr/bin:/bin"
  local e, n
  journal("      % " & command)
  try
    do shell script command
  on error e number n
    journal("      ! " & (n as string) & " " & e)
    error e number n
  end try
end shell

on wait(interval)
  shell("sleep " & interval as string)
end wait

-- ismounted, mount, unmount : mounting and unmounting volumes

on ismounted(volname)
  try
    -- if not found, egrep exits with a non-zero status
    shell("/sbin/mount | egrep 'on /Volumes/" & volname & " \\(.+\\)$'")
  on error
    journal(" - " & volname)
    return false
  end try
  journal(" + " & volname)
  return true
end ismounted

on mount(volname)
  local a
  journal("mount: " & volname)
  if not ismounted(volname) then
    try
      -- if not found, grep and egrep exit with a non-zero status
      set a to shell("/usr/sbin/diskutil list | egrep '^ +[[:digit:]]+: +[[:graph:]]+ +" & volname & " +[.[:digit:]]+ GB +disk[[:alnum:]]+$'")
      set a to shell("echo '" & a & "' | sed 's/" & volname & "//' | awk '{print $5}'")
      shell("/usr/sbin/diskutil mount " & a & " | grep '^Volume " & a & " mounted$'")
    on error
      return false
    end try
  end if
  return true
end mount

on unmount(volname)
  local n
  journal("unmount: " & volname)
  if ismounted(volname) then
    try
      -- unmount always generates error "Volume failed to unmount", even when successful
      shell("/usr/sbin/diskutil unmount '/Volumes/" & volname & "'")
    end try
  end if
end unmount

-- mountVol, unmountVol : patient, stubborn wrappers for mount, unmount

on mountVol(pause, tries, volname)
  local n
  journal("mountVol " & volname)
  repeat with n from 1 to tries
    wait(pause)
    if mount(volname) then
      repeat with n from 2 to 8
        wait(n)
        if ismounted(volname) then return true
      end repeat
    end if
  end repeat
  return false
end mountVol

on unmountVol(pause, tries, volname)
  local n
  journal("unmountVol " & volname)
  repeat with n from 1 to tries
    wait(pause)
    unmount(volname)
    repeat with n from 2 to 8
      wait(n)
      if not ismounted(volname) then return true
    end repeat
  end repeat
  return false
end unmountVol

-- mountVolumes, unmountVolumes : apply mountVol, unmountVol to lists

on mountVolumes(pause, tries, volnames)
  local status
  journal("mountVolumes")
  set status to true
  repeat with volname in volnames
    if not mountVol(pause, tries, volname) then set status to false
  end repeat
  return status
end mountVolumes

on unmountVolumes(pause, tries, volnames)
  local status
  journal("unmountVolumes")
  set status to true
  repeat with volname in volnames
    if not unmountVol(pause, tries, volname) then set status to false
  end repeat
  return status
end unmountVolumes

-- SuperDuper!

on SuperDuperJob(volname, scriptalias)
  local wasmounted
  journal(scriptalias as string)
  journal("")
  set wasmounted to ismounted(volname)
  if wasmounted or mountVol(30, 4, volname) then
    journal("SuperDuperJob: " & scriptalias as string)
    try
      run script scriptalias
    on error e number n
      journal("script error: " & (n as string) & " " & e)
    end try
    if not wasmounted then unmountVol(30, 4, volname)
  end if
  journal("")
end SuperDuperJob

-- top level routines

on mountBackups()
  mountVolumes(0, 1, {
    "aOSX", "aOS9", "aOSX 10.4.4 min", "aOSX 10.3.9 min", "aUser", 
    "bOSX", "bOS9", "bOSX 10.4.4 min", "bOSX 10.3.9 min", "bUser", 
    "FireWire A OSX", "FireWire A"})
end mountBackups

on unmountBackups()
  unmountVolumes(0, 1, {
    "aOSX", "aOS9", "aOSX 10.4.4 min", "aOSX 10.3.9 min", "aUser", 
    "bOSX", "bOS9", "bOSX 10.4.4 min", "bOSX 10.3.9 min", "bUser", 
    "FireWire A OSX", "FireWire A"})
end unmountBackups

on runBackups()
  journal("runBackups")
  
  SuperDuperJob("aOS9", 
    alias "User:Users:ad:Scripts:Smart Update aOS9 from OS9.sdsp:Copy Job.app:")
  SuperDuperJob("aOSX 10.3.9 min", 
    alias "User:Users:ad:Scripts:Smart Update aOSX 10.3.9 min from OSX 10.3.9 min.sdsp:Copy Job.app:")
  SuperDuperJob("aOSX 10.4.4 min", 
    alias "User:Users:ad:Scripts:Smart Update aOSX 10.4.4 min from OSX 10.4.4 min.sdsp:Copy Job.app:")
  SuperDuperJob("aUser", 
    alias "User:Users:ad:Scripts:Smart Update aUser from User.sdsp:Copy Job.app:")
  SuperDuperJob("aOSX", 
    alias "User:Users:ad:Scripts:Smart Update aOSX from OSX.sdsp:Copy Job.app:")
  
  SuperDuperJob("aBack", 
    alias "User:Users:ad:Scripts:Smart Update Sources image from User.sdsp:Copy Job.app:")
  SuperDuperJob("aBack", 
    alias "User:Users:ad:Scripts:Smart Update Users image from User.sdsp:Copy Job.app:")
end runBackups
This is a typical AppleScript application file that makes a call from the above library, Run Backups.app. Once set up, it seldom needs further editing. Opening it runs the script:

Code:
set BackupsLib to (load script file "User:Users:ad:Scripts:Backups Library.scpt")

tell BackupsLib
  runBackups()
end tell
This is a typical journal entry, from the file log 2006-01-22.txt:

Code:
unmountVol aBack

07:11:37         % sleep 0
07:11:37   unmount: aBack
07:11:37         % /sbin/mount | egrep 'on /Volumes/aBack \(.+\)$'
07:11:37    + aBack
07:11:37         % /usr/sbin/diskutil unmount '/Volumes/aBack'
07:12:04         ! 1 Volume failed to unmount
07:12:05         % sleep 2
07:12:07         % /sbin/mount | egrep 'on /Volumes/aBack \(.+\)$'
07:12:07         ! 1 The command exited with a non-zero status.
07:12:07    - aBack
Reply With Quote
  #4  
Old 02-08-2006, 10:19 AM
Syzygies Syzygies is offline
Registered User
 
Join Date: Jan 2006
Posts: 23
Hi. Has anyone tried this script?

Reply With Quote
  #5  
Old 02-08-2006, 10:22 AM
dnanian's Avatar
dnanian dnanian is offline
Administrator
 
Join Date: Apr 2001
Location: Weston, MA
Posts: 14,290
Send a message via AIM to dnanian
Any particular reason for the graphical sig, Dave? Tracking?
__________________
--Dave Nanian
Reply With Quote
  #6  
Old 02-08-2006, 11:07 AM
Syzygies Syzygies is offline
Registered User
 
Join Date: Jan 2006
Posts: 23
Sure, most forums are 99% lurkers, and the code took a bit of work, just curious. I see how many people read my post, and they can study source to figure out who I am.

Here's a simple Perl script that can extract this kind of information from my apache server. No idea how portable the code is, but it works for me:

Code:
#!/usr/bin/perl
use warnings;
use strict;

# apachelog.pl  --  Perl script for extracting reports from /usr/local/apache/logs/access_log
#
# Copyright (c) 2006 Dave Bayer
# Subject to the terms and conditions of the MIT License.

(my $help = <<'EOF') =~ s/^: ?//mg;
:
: Usage:
:
: 	apachelog.pl count path
:		count distinct accesses
:		(in all, how many distinct visitors?)
:
: 	apachelog.pl who path
:		list distinct accesses by caller
:		(who were the visitors?)
:		
: 	apachelog.pl when path
:		list distinct accesses by count for each date
:		(for each date, how many distinct visitors?)
:		
: 	apachelog.pl what path
:		list distinct accesses by ascending count
:		(for each distinct visitor, what did they access?)
:
:	apachelog.pl first path
:		list distinct first accesses by ascending count
:		(for each distinct visitor, what did they access first?)
:
: Examples:
:
:	apachelog.pl what ''
:	apachelog.pl when bayer/coffee
:
EOF

my $get = ' "GET /(?:~|%[0-9a-fA-F]{2})*';

sub openlog {
	open LOG, "</usr/local/apache/logs/access_log" or die "unable to read /usr/local/apache/logs/access_log\n";
}

sub convertdate {
	local $_;
	($_) = @_;
	
	s/(J|j)an[a-z]*/01/;
	s/(F|f)eb[a-z]*/02/;
	s/(M|m)ar[a-z]*/03/;
	s/(A|a)pr[a-z]*/04/;
	s/(M|m)ay[a-z]*/05/;
	s/(J|j)un[a-z]*/06/;
	s/(J|j)ul[a-z]*/07/;
	s/(A|a)ug[a-z]*/08/;
	s/(S|s)ep[a-z]*/09/;
	s/(O|o)ct[a-z]*/10/;
	s/(N|n)ov[a-z]*/11/;
	s/(D|d)ec[a-z]*/12/;
	s:(\d*)/(\d*)/(\d*):$3-$2-$1:;
	$_;
}

sub who {
	my ($match) = @_;
	my (%who, $addr);

	openlog();
	while (<LOG>) {
		/$get$match/ or next;
		($addr = $_) =~ s/^(\S*)\s.*\n/$1/;
		$who{$addr}++;
	}
	close LOG;
	%who;
}

sub when {
	my ($match) = @_;
	my (%when, $date, $addr);

	openlog();
	while (<LOG>) {
		/$get$match/ or next;
		($date = $_) =~ s/^[^[]*\[([^:]*):.*\n/$1/;
		($addr = $_) =~ s/^(\S*)\s.*\n/$1/;
		$when{convertdate($date)}{$addr}++;
	}
	close LOG;
	%when;
}

sub what {
	my ($match) = @_;
	my (%what, $file, $addr);

	openlog();
	while (<LOG>) {
		/$get$match/ or next;
		($file = $_) =~ s/^.*?$get($match\/?[^&#%\?\/\s]*).*\n/$1/;
		$file =~ s/\/$//;
		($addr = $_) =~ s/^(\S*)\s.*\n/$1/;
		$what{$file}{$addr}++;
	}
	close LOG;
	%what;
}

sub first {
	my ($match) = @_;
	my (%start, %first, $file, $addr);

	openlog();
	while (<LOG>) {
		/$get$match/ or next;
		($file = $_) =~ s/^.*?$get($match\/?[^&#%\?\/\s]*).*\n/$1/;
		$file =~ s/\/$//;
		($addr = $_) =~ s/^(\S*)\s.*\n/$1/;
		if (defined $start{$addr}) { next; }
		$start{$addr} = $file;
	}
	close LOG;
	foreach $addr (keys %start) {
		$first{$start{$addr}}++;
	}
	%first;
}

sub count_cmd {
	my %who = who(@_);
	printf "%d\n", scalar keys %who;
}

sub who_cmd {
	my %who = who(@_);
	foreach my $addr (sort keys %who) {
		print "$addr\n";
	}
}

sub when_cmd {
	my %when = when(@_);
	foreach my $date (sort keys %when) {
		printf "%8d  %s\n", scalar keys %{$when{$date}}, $date;
	}
}

sub what_cmd {
	my %what = what(@_);
	my @list;
	foreach my $file (sort keys %what) {
		push @list, sprintf "%8d  %s\n", scalar keys %{$what{$file}}, $file;
	}
	print sort @list;
}

sub first_cmd {
	my %first = first(@_);
	my @list;
	foreach my $file (sort keys %first) {
		push @list, sprintf "%8d  %s\n", $first{$file}, $file;
	}
	print sort @list;
}

my %run = (
	'count' => \&count_cmd,
	'who' => \&who_cmd,
	'when' => \&when_cmd,
	'what' => \&what_cmd,
	'first' => \&first_cmd,
);

my ($do, $match) = @ARGV;
if ($#ARGV != 1 or not $run{$do}) { 
		print $help;
		exit(1);
}
$run{$do}($match);
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems automating backups with applescript and crontab kbradnam General 4 01-20-2006 07:40 AM
Strongspace SuperDuper backup strategy options? parakeet General 1 12-21-2005 09:16 PM
Backup and archive strategy and method? xoshooter General 3 10-07-2005 04:28 PM
applescript help snoopy67 General 1 09-17-2005 09:07 AM
AppleScript bug? fixed! pzingg General 1 11-02-2004 06:37 PM


All times are GMT -4. The time now is 05:20 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.