Shirt Pocket Discussions

Shirt Pocket Discussions (http://www.shirt-pocket.com/forums/index.php)
-   General (http://www.shirt-pocket.com/forums/forumdisplay.php?f=6)
-   -   Sparse image from aborted DMG backup? (http://www.shirt-pocket.com/forums/showthread.php?t=754)

Winston 10-20-2005 10:40 PM

Sparse image from aborted DMG backup?
 
I tried to back up a drive containing about 7 GB to a drive that had about 12.5 GB free. To make the final DMG disk image SD! first makes a sparse image, then makes a DMG image from the sparse image. There are two backup images until SD! finishes - taking up twice the disk space.

Most of the way into the backup I realized that SD! would not have enough room to finish. Setting the backup to maximum compression did not help enough.

I stopped the backup and trashed the incomplete DMG image. I opened the sparse image that was left. I had what looks like a complete backup.

1. Is the sparse image a complete backup?

2. Can I use the sparse image for incremental backups as described in the FAQ for creating a "modifiable" disk image?

3. Can SD! warn about backups that won't work because there is not enough disk space?

4. Copying from the sparse image to the DMG image is slow. If the sparse image and the DMG image could be created on separate drives the copying would be a lot faster. Is there a way to do this? If the sparse image is made on the original drive then making it will be really slow, and you don't want the DMG image on the original drive. Would a third drive allow a disk image backup to be faster?


- Winston

dnanian 10-20-2005 10:50 PM

Let me try to answer:

1. Yes, it's the full backup that was present before conversion.

2. Yes.

3. No, it can't without processing all the files first. Since you'd have to pay that cost for every backup, rather than just the occasional out-of-space failure, it didn't seem to be worth the penalty. (It's also not a guarantee, because the drive can change during processing...)

4. At present, there's no way to place the two parts of the image process on different drives. But, it's unlikely to be significantly faster... we do have some tricks up our sleeves in v2.0, though, that should speed things up.

Winston 10-20-2005 11:05 PM

Thanks for the answers - that's a big help. (and will no doubt save me a lot of time)

Quote:

Originally Posted by dnanian
Let me try to answer:
3. No, it can't without processing all the files first. Since you'd have to pay that cost for every backup, rather than just the occasional out-of-space failure, it didn't seem to be worth the penalty. (It's also not a guarantee, because the drive can change during processing...)

As I mentioned on the thread about backing up to CDs, it would be easy to check for space available after the sparse image is made, but before the DMG image is started.

The advantage would be that instead of having a complete failure of the backup, there would still be the sparse image. A full backup in a slightly "off" format seems far better than no backup.

SD! could have an option to convert the sparse image to a DMG later, when more disk space is available.


Thanks for your patience with me!

- Winston

dnanian 10-20-2005 11:22 PM

Thanks for the suggestion, Winston. But, again, we just *don't know* how much space things are going to take. Compression can act differently on every image, and trying to guess means we'll get it wrong as much as we get it right.

Yes, we could probably do it for uncompressed images. But, given that's just one setting, and not the default, it just doesn't seem worth it...

Winston 10-21-2005 05:27 PM

Space estimate for disk image backup?
 
Quote:

Originally Posted by dnanian
Thanks for the suggestion, Winston. But, again, we just *don't know* how much space things are going to take. Compression can act differently on every image, and trying to guess means we'll get it wrong as much as we get it right.

Yes, we could probably do it for uncompressed images. But, given that's just one setting, and not the default, it just doesn't seem worth it...

Now I am confused! Is this correct for making a DMG backup:

1. SD! makes a complete copy of the original to a sparse image. So, with the exception of any space saved by de-fragmenting the files on the orignal drive, the sparse image takes up the same amount of disk space as the original files.

2. The sparse image is not compressed. But it is a complete copy that ccould be used to restore the original. The sparse image can also be used for incremental backups.

3. If the user set up SD! to do compression during the backup the compression takes place while converting the sparse image to the DMG image. Because of this, the exact size of the DMG image cannot be predicted exactly, even when the size of the sparse image is known.

4. If there is no compression the DMG image is the same size as the sparse image.

5. Once the DMG image is complete, SD! removes the sparse image. However, if the backup is stopped after the sparse image is finished, but before the DMG image is done, the sparse image is left in place.

Assuming the above is correct -
Even if SD! cannot estimate exactly how large the final DMG image will be it should be able to estimate the size of the (uncompressed) sparse image. At a minimum, it would help me to get a warning that there may not be enough space on the backup drive to complete the DMG creation.

Even if estimating the sparse image size is too difficult it would help to have a warning any time backup to a disk image is requested - it is the only type of backup SD! does that might fail because of a lack of disk space on a drive that has a lot more available space than the original.

Note that while there is one warning in the User's Manual about the space needed to do a disk image backup, it is not obvious.

A search of the User's Manual for "disk image" gives 6 references, on pp. 23,29 and 30. None of these pages contain the warning. A search for "image" turns up what must be over a hundred references. The relevant one is the fifth reference to p. 21:
"the original image, and the"

Not an obvious link.

Here is the sentence, from the middle of p.21:
"This operation requires approximately twice the disk space taken by the original image, and the conversion can take a few hours to process."


Not having a backup because of not enough disk space seems worse to me than having to get a warning, especially since the failed backup may not happen for several hours (since SD! goes through most of the process before the backup fails).

Think about someone about to leave on a trip, who only has time for one backup run. (This has happened to me.) When it fails, no backup.

I came to SD! from Carbon Copy Cloner. It always worked, even if it had fewer options. I think CCC only makes sparse image backups when a disk image backup is asked for. Maybe this is why I was surprised at the difficulty in doing this with SD!.


Thanks.

- Winston

dnanian 10-21-2005 05:41 PM

Winston: the only way to estimate the size of the sparse image is to actually go through all of the files on the source, select the ones that will be selected by the script, and add up the sizes. Doing so, before each backup, would be extremely time consuming every time the backup takes place. That's why we don't do it.

A warning might be useful, once. But typically this kind of thing is ignored after the first time (if it's read at all the first time through)...

Again, we're improving this in v2.0. and the default will be to create a sparse image, which won't take 2x the space... but we won't, at least in v2.0, be calcuating whether or not the backup will fit on the destination.

That said, I'm willing to revisit the decision later on, as time permits...

Winston 10-21-2005 06:47 PM

What I am thinking about is complete (clone) backups to a disk image. Don't these take up about the same space as the original files?

- Winston

dnanian 10-21-2005 07:06 PM

Yes. But we don't know whether or not a "complete backup" is "complete": a script name is just text, and doesn't have any meaning to SD! itself...

mry 10-22-2005 12:12 PM

Quote:

Originally Posted by dnanian
Winston: the only way to estimate the size of the sparse image is to actually go through all of the files on the source, select the ones that will be selected by the script, and add up the sizes. Doing so, before each backup, would be extremely time consuming every time the backup takes place. That's why we don't do it.

A warning might be useful, once. But typically this kind of thing is ignored after the first time (if it's read at all the first time through)...

Again, we're improving this in v2.0. and the default will be to create a sparse image, which won't take 2x the space... but we won't, at least in v2.0, be calcuating whether or not the backup will fit on the destination.

That said, I'm willing to revisit the decision later on, as time permits...

lol I downloaded SD! and used it for the first time today. Got into the same predicament as Winston did. I selected Bakup-User Profile option on my mac mini disk. The target was a FW disk with 30GB space left. After 1.5 hours, the backup failed and I was left with the sparse disk image.

Anyhow, I didn't read the manual that carefully and was a bit surprised to see the two step procedure. I am excited about the clone feature and will try that out later.

/mry

dnanian 10-22-2005 12:28 PM

Thanks for the feedback, /mry. You might want to consider using a sparse image, too -- see the FAQ ("How do I update an image?") for the technique -- no conversion necessary.


All times are GMT -4. The time now is 05:32 AM.

Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.