PDA

View Full Version : Restoring to a folder


ericob
10-14-2005, 05:54 PM
I want to back up a small(~14GB used) volume, then restore its contents to a folder. Unless I'm mistaken, there is no way to restore to a folder when using SuperDuper.

Do I have any better options than backing up to an image, mounting the image and then copying the contents of that image into a folder? Using the Finder to then copy tens of thousands of files seems very unpleasant.

Is there some nice little terminal command or utility that would be a better choice for this copy? Is there some nice little terminal command or utility that would be able to read directly from the disk image file and copy into a folder I designate?

dnanian
10-15-2005, 08:52 AM
Why are you trying to store its contents in a folder, Eric? Would a sparse image suffice, since it can be opened and managed like any other volume?

If you're just going to copy out of the image into a folder, you could do the *whole* operation with Finder... but since you probably can't authorize to copy that much data, it's unlikely to work...

ericob
10-15-2005, 06:00 PM
There are several issues embedded in my comment.

First, to reply: It is not that I want to "store" the contents of a disk image in folder. I want to REstore a disk image TO a folder. Currently, we can backup volumes to images and folders to images. And we can restore images to volumes. But we cannot restore an image to a folder. Realize than an image will not always contain a bootable volume: it may contain only a folder (a collection of files), just a single very large file, a data-only volume, etc.

I realize I can open an image and use the Finder to copy everything (or pick and choose) to another location. That will often be useful, but also is not what I'm looking for.

I really do not want to have to rely on the Finder to copy these files. And I'd prefer to not even let the Finder see the files before they appear in my target folder. (I don't want Spotlight or some other "service" grabbing for things on a mounted image.) Also, there may be invisible files on the image. A mounted image will not show those files in the Finder, and therefore they cannot be selected for copying. For another thing, I want all the file information preserved during the process. I do not want permission settings or modification dates of the files changing, for instance. It's also difficult to believe that the Finder is going to be very efficient copying thousands and thousands of files. And how good it its validation checking?

As I say, I'd prefer a solution that will restore the contents of a disk image file to a folder (without needing to mount that image). As a second choice I'll settle for mounting the image and using a command or tool in the terminal to do the copy task. Which one(s) will preserve all the "Macintosh stuff" in files?

In your reply you wrote "...but since you probably can't authorize to copy that much data...." I've never heard of this sort of limit. Are you saying that the Finder will refuse to copy more than a certain number of files at a time??


In case anyone wonders why I would want to keep a collection of file in a disk image (instead of simply in a copy of the folder), it is because in an image the files are "hermetically sealed." I can safely have a disk image visible to my file system without exposing its contents to the file system. I won't mistakenly begin working inside my "backup" folder, iTunes or iPhoto or Spotlight or WHATever will not notice the contained files and feel bound to do something with them, etc. Also, again, the image should be an exact copy of my files, not the simply the Finder's idea of what the current user can see or do.

dnanian
10-15-2005, 07:31 PM
First, I used the wrong term -- sorry. I didn't mean "much" in quantity terms, I meant for permissions -- trying to copy a whole system, and authorizing against the whole thing.

But, regarding the rest -- sorry. We can't do that... and even if we could, they'd be seen by Finder anyway, since you'd have to mount (or we would)... and Spotlight sees the files as soon as they hit the file system, since that's all done with kqueues.

Block restore -- the only real way to avoid mounting -- couldn't be done to a folder either, because it happens at far too low a level. You'd have to replace the *entire* drive with the restoration... and then it gets seen, and indexed. Etc.

So, unfortunately, we don't look to be the solution to your needs. Sorry, Eric!

ericob
10-15-2005, 09:59 PM
First, I used the wrong term -- sorry. I didn't mean "much" in quantity terms, I meant for permissions -- trying to copy a whole system, and authorizing against the whole thing. Gotcha. Note that I'm not intending to (try to) copy a entire bootable system by dragging files in the Finder. (To copy a bootable volume I'd of course use SuperDuper! :) ) But neither do I expect to be able use the Finder to drag around files that are owned by other users (the System included). However, I can get around the permissions problems at the point of creating the image... I use SuperDuper!

But, regarding the rest -- sorry. We can't do that... and even if we could, they'd be seen by Finder anyway, since you'd have to mount (or we would)... and Spotlight sees the files as soon as they hit the file system, since that's all done with kqueues.

Block restore -- the only real way to avoid mounting -- couldn't be done to a folder either, because it happens at far too low a level. You'd have to replace the *entire* drive with the restoration... and then it gets seen, and indexed. Etc.Well, I didn't know the limitations of reading from an image directly. It appears that there's no way to just reach in there and pull files out, one at a time, sequentially. And you write "You'd have to replace the *entire* drive with the restoration." That's exactly what I'm trying to avoid. Hence the concept of "restoring to a folder." :( Apparently, if the backup file was some form of archive file (instead of an image file) I could then copy out of the archive to another destination without exposing the files inside the archive directly to the file system. Hmm...

So, unfortunately, we don't look to be the solution to your needs. Sorry, Eric! Not entirely... I can get the data backed up completely and accurately (to a disk image) using SD. To then copy the contents of that image to a folder, it looks like I will have to mount the image (ugh), then do the copy operation from the Terminal with "sudo cp" or some such so that I can completely and accurately restore all the files, including invisible ones and ones I don't own.

*sigh* Not exactly one-stop shopping.

dnanian
10-15-2005, 11:30 PM
Well, you can reach inside an image and pull files out, but since it's a volume like any other, it gets mounted and used. The *advantage* to an image is that it's just like any other volume, and restoration is easy. You have rather unique requirements here, which is why it seems to be a problem... but I truly don't understand the problem with "exposing" the files to the file system, because it can't really do any "harm"... what could possibly happen?

ericob
10-16-2005, 08:02 PM
It is more the potential for irritation than concern for harm (although there is that possibility too). Perhaps I'm being superstitious, but (during a restore) I'd just as soon not give the OS and any of it's "automatic" systems something else to play with. I realize the image will be mounted only for a little while, but it irritates me to think that Spotlight (or whatever) might start (pointlessly) scanning it.

As to harm, there is a potential for harm because the mounted image is writeable. Of course the image was created read/write because, well... you couldn't very well back up to it if were read only!

However, when "ejected" any changes, whether accomplish by me (by mistake) or some mystery or rogue process will be, without any notification of course, written to the image file! While this is unlikely to happen, there is nothing to prevent if from happening. Unlike if the backup files were in an archive or an unmounted image file.

One can "convert" a read/write sparse image file to a read only .dmg file, but this is yet another step, and something one must remember to do. Irritatingly, the conversion isn't just a switch you can flip to lock the existing image... an entirely new one is written.

dnanian
10-16-2005, 08:25 PM
I guess, Eric, but -- in my experience -- this just doesn't seem to be a problem. Of course, there's always a chance of trouble -- after all, even a given archive can be scanned, since it's a file. And the data in it would be scanned... and the archive could get damaged, and you'll be looking at another level of indirection, which means things are potentially even more fragile... I guess it depends what you want to worry about! :)

ericob
10-22-2005, 02:08 AM
[FYI, it is not that I consider potential scanning a threat to the integrity of my files... it is that the image is mounted read/write, and in that condition anyone or anything can mess with the data, and that messing-with will be saved without any notification when you unmount the image. A simple "slip of the wrist" here could erase the entire contents of your disk image. Leaving the image intact, but empty. Something as critically important as data from a backed up disk should never be left this vulnerable.]

However, regarding the main thread: I believe I have found the solution to restoring a backed up volume to a folder.

This little adventure is my current winner for "Far, Far More Difficult than it Needs to Be (or even Is)."


Recall that my problem was how to "reshuffle" the contents of partitions and drives on my system. At my (apparent) success point, I have gone from 5 drives and 14 partitions down to 4 drives and 7 partitions. Probably still too many, but this is FAR more manageable!

The sticking point in this was my desire to back up a volume (a partition) and restore it to a FOLDER.

SuperDuper! can't do this.
CarbonCopyCloner can't do this.
Impression (a backup utility) appeared to be doing it, but screwed up the modification dates (and who knows what else) of not all, but most, of the files restored. It also restored them to a bogus "/Volumes" folder inside the folder I specified. Plus, the "archive file" is actually just a package with every single file on the volume (but none of the folders) individually stored in an archive. I can't imagine that's very efficient.

PLEASE, if anybody reading this believes that I may be making a Really Big Mistake, do let me know! But the answer appears to be to use the built-in command line utility "ditto" running as root.

Under 10.4 ditto by default preserves resource forks -- when using a OS X version earlier than 10.4 you must explicitly use the switch "--rsrc" or "-rsrcFork" ("Preserve resource forks and HFS meta-data").

SO. Consider that I have a partition [or a mounted disk image] named "Downloads" whose entire contents I want to move to a FOLDER named "Downloads," which exists on a different volume. I want the "cloned" files to be identical (as near as possible) to the originals. Specifically: I want the creation and modification dates and the permissions of every "copied" file to be identical to the original. Of course, I also want all meta-data preserved.

Hey, it's easy! :D

The form is

ditto source_dir destination_dirI want to avoid any problems copying files I don't own, so I run ditto as root:

sudo ditto /Volumes/Downloads /Volumes/chinesemenu/Downloads("chinesemenu" is a new volume I recently created) Presto! 30 minutes or so later, everything is copied. By the way, Impression took about 4 HOURS to restore the same volume.

ditto apparently also copies invisible files. This is both good... and problematic. I don't think I need to (and in fact, maybe I shouldn't) copy those hundreds of .DS_Store files. What about the .Trashes files? The "501, 502 and 503" folders? An empty "Cleanup at Startup" folder? AppleShare PDS, Desktop, Desktop DB and Desktop DF files? A folder named "EU2P9I1J4FI58RJ8NMDF9901C8" contained in a folder named "HFSExtentTables" (Created and Modified December 12, 2002)?? Etc., etc., etc. (You can see I've had this drive, or at least its data, around for some while!)

If anyone has any suggestions about invisible files I most certainly should find and delete, please let me know.

If anyone knows of a way to tell ditto to skip files matching a particular criterion, please let me know too.

And, like I said, if I'm really making a Big Mistake using ditto for this, please tell me about that, too. :)

Thanks!

dnanian
10-22-2005, 10:05 AM
Well, the thing is that a lot of the volume has no meaning in a folder. Many of the files you're talking about are system files, but you no longer have a valid system... in the end, if you're just trying to keep the data in a folder, I wouldn't delete anything, since the space is probably less valuable than what might be deleted by making a mistake.