why averaging only 4MB/s and how to spare bad blocks?

[sjs]

I am happy to report that this method works amazingly well.

Check out the space savings on the destination:

-rw-rw-r-- 1 demo demo 6381662208 Mar 20 21:25 demo.sparseimage
-rwx------ 1 demo demo 9484529664 Mar 17 06:12 demo.sparseimage.orig

now as you can see the only problem is i needed to
chmod 700 demo.sparseimage, but i would imagine SD can do this in a post-script i think...

the amazing this is the old demo.sparseimage.orig, which is about the same size as my source sparseimage is so big despite being routinely compacted by Finder logout.

so doing a fresh copy really is the ultimate compaction or a sparseimage. Zero bloat.

SD didnt eliminate any useless files did it? like Caches or things it might not copy if a root copy was in place? I doubt it would do that, but I could do a diff against the 2 file sets....

Also it was quite speedy. Does the algorithm change when copying to local HDs vs the network? Are you tuned for slower links like rsync?

In any event, I am convinced that filevaults are nothing more than sparseimages. In fact, the method I have been using to date for over 4 years and 5 computers, is I keep moving my sparseimage file from machine to machine over the years (I've been installing OS X, create my user accts in a particular order to preserve UIDs, enabling Filevault on the acct and finally substituting my old sparseimage from the cmd line. It has been working well for years and I'm on my original homedir since 10.0). Perhaps there is a better way now that I am using SD, I havent gotten into all its nooks and crannies what it can do...

[sjs]

whats also interesting, is that when the disk image is mounted:

Finder > Get Info reveals: 7.71GB ( byte size notwithstanding)

and yet

$ dh -kh
$ 5.1G .

and well as:
$ ls -lh
-rwx------ 1 demo demo 5G Mar 21 01:06 demo.sparseimage

quite an unexpected difference to me. 2.6GB or 2.7GB less data then I thought I had. Quite amazing.

At first I thought 1/3 of my data was missing or SD failed, but thats not the case. Just a pleasant surprise.

Thinking about it now, I think this is probably due to the cluster size within the sparseimage, as I have tons of small text files in there. I wonder if I can make the cluster size 1K or so on a sparse image.

Either way this make SD a great way to copy filevaults for me.

I would have to suspect, however, that the SD benefit is only the 1st time. If I keep doing a Smart Update, I'd guess I'd quickly fragment up and grow this image too. I'd imagine that even an "Erase, then Copy Files" would not shrink the sparse image would it? I'd imagine even if you did a format to the volume when you do your "Erase" (do you?) it would not shrink unless your Erase really deletes or compacts the image (not a bad idea if you dont)...
Just some thoughts...

[dnanian]

OSX will actually compact FileVaults automatically, so it should work automatically post-bloat.

We haven't done any thing differently in network backups, but they've always been pretty reasonable given a reasonable connection... and, no -- files shouldn't be missing.

[sjs]

hi, thanks,
i dont know what you mean with this line

"OSX will actually compact FileVaults automatically, so it should work automatically post-bloat."

are you saying that after a destination sparse image gets heavily fragmented and bloated, a simple deletion of all files automatically triggers os x to compact any given sparseimage? what is the trigger? a certain threshold of deletions or too much fragmentation? i don't think i am understanding what you are saying... i thought the trigger was a Log Out User, and it only applied to File Vaults, not sparse images destinations that SD creates that aren't subjected to any Log Out process...

can you talk about what an Erase than Copy does? Does it format first or just delete files first?

[dnanian]

It is only through log out, but since this is a FileVault, I figured it would happen for you when you really need it -- in place...