Ever-Growing Sparse Image File...
 

Running Mac OS X 10.4.8 on a Quicksilver with 1.5GB RAM...

I am backing up a 250GB drive to a Linux-based NAS using a sparse image as the target. The NAS drive is 320GB. The backup uses Backup All Files with Smart Updating and is scheduled to run on a nightly basis.

With each backup, the sparse image grows in size until it fills up my NAS.

A backup of 180GB from a 250GB drive can end up taking up 320GB?

My understanding of the way that sparse images work suggests that the image should grow as necessary, and then when files are deleted, it simply does not shrink.

I do a lot of video work and the contents of my drive changes from day-to-day to the tune of up to 20GB worth of data. Even so, there is never more than 180GB on that drive during the nightly backup.

I can see that if the new data files are copied to the sparse image before the old ones are deleted the image might end up as much as 20GB larger than the contents of the original drive. That might make for a 200GB disk image... But it's presently at 233GB and is threatening to outgrow my NAS for the third time this month. (I've deleted 2 images and started from scratch twice when the NAS filled up.)

I can't tell what's taking up the extra space.

When I mount the disk image, it appears to be a FULL 233GB volume, but when I look to see if my old video files are present on the image, they are not there. I also searched the image with FileBuddy for files greater than 1GB in size and that's showing no unexpected files.

Looking at the log, I see no errors. At the end of the clone, this is what it reports:

Evaluated 643207 items occupying 172.65 GB (99699 directories, 532444 files, 11064 symlinks)
Copied 11863 items totaling 20.30 GB (183 directories, 436 files, 11244 symlinks)
Cloned 172.17 GB of data in 17750 seconds at an effective transfer rate of 9.93 MB/s

1. Have you got any idea what's going on and how to correct it?

2. In the alternative, is there any way for me to make a scheduled back up to a regular .dmg disk image file?

I tried creating a 200GB disk image on the NAS and then scheduling a backup onto that, but instead of mounting and copying to that disk image, SuperDuper created a new sparse image file and attempted to clone onto the sparse image.

This seems to be the way Apple has implemented the sparse image handling. It doesn't always re-use free space, and the images never shrink (they just grow as necessary).

You can shrink the image yourself, though. Here's what to do:

• Start Terminal (/Applications/Utilities/Terminal)
• Type (in Terminal)
  Code:

  ---

  hdiutil compact

  ---

  with a space after it.
• Drag in the image file and drop it on the Terminal window
• Press Return

This'll run for a while and shrink the image file down to the size of the data inside.

'Problem is, it takes hours and hours (and hours) to shrink the sparse image that way.

And I'd have to automate it to run pretty often.

And doing that on top of doing the nightly backup would mean that my machine would be busy with the backup and compacting of the image well into daylight work hours.

It would probably take less time to erase the current backup and start from scratch every night.

Is there any easy way to work the alternative? To back up to a regular disk image every night? SuperDuper refuses to mount the disk image when I try this -- it creates a new sparse image rather than cloning to a .dmg file.

I suppose that I could have it run a bit of AppleScript to mount the disk image first... is that the best way to get it to make a scheduled clone to a regular disk image?

Sure, you can create a standard Read/Write image and have us write to it. Don't select it in the Disk Image pop-up -- instead, point us to the already-mounted volume.

We've been hoping that Apple's sparse image filesystem code will improve its free space handling for some time...

Thanks!




`

Correction
 

The command is

with a space after it.

There is a typo in the version above :)

Yep, fixed. thanks.