PDA

View Full Version : questions about copying of very large (32GB) file


mykmelez
12-15-2006, 11:00 PM
I run a Parallels VM with a virtual drive that lives as a single very large (currently 32GB) file on my hard drive. SD appears to stall for a long time in the middle of a Smart Backup (files evaluated, effective copy speed, etc. statistics stop changing), and I suspect that it's just copying that very large file.

I have a few questions about that:

If anything changes in a file, does SD copy over the entire file or just the changed bits? If so, has any thought been given to implementing the latter? If would significantly improve performance in my use case, since much of the stuff on my virtual drive doesn't change between backups.

Or, if that's not practical or feasible, might stats be made available during the copying of a large file (f.e. the name of the file and how much has been copied) to stop me from fretting about SD's state?

Finally, if I run SD while I'm using my virtual machine, what'll happen when SD reaches that file? Will it lock it for copying, freezing my VM; stall waiting for my VM to finish with it; or copy it concurrently, potentially leaving the backup in an inconsistent state?

dnanian
12-15-2006, 11:23 PM
That's right, it's copying the whole file. It has to copy the entire file every time: there's no way -- without reading the entire file and looking -- to see if "part" of it has changed. That's just not how things work, and reading/comparing two huge files would take as long as just copying it.

At present, the APIs we're using to copy files do not give any status during a file copy, so we have no way of letting you know how much of a file is copied, sorry.

Don't run SD! while your VM is running. It'll likely end up filling the destination... bad idea.

mykmelez
12-15-2006, 11:31 PM
At present, the APIs we're using to copy files do not give any status during a file copy, so we have no way of letting you know how much of a file is copied, sorry.

Ok, good to know. FWIW, in a "related thread", I found this technique you posted for figuring out what file is being copied:

run the following in Terminal:

sudo lsof | grep SDCopy

The last two files listed are what we're reading and writing.

Using this technique, I was able to confirm that SD! is indeed copying my large virtual drive file when it appears to stall.

Don't run SD! while your VM is running. It'll likely end up filling the destination... bad idea.

Oof, sounds bad. Ok, I'll avoid that.