PDA

View Full Version : SD v2: *Special Spotlight handling to improve performance


Hoosier_1701
11-23-2005, 10:44 AM
I saw this entry in the version history for v2.0. Can you give us some details on what this means? Using the previous version, I've been rebooting my Mac with the Shift key held down to completely disable Spotlight during my backup sessions. Once the backup finished, I would then reboot again to restore normal functionality. Is this unnecessary now with v2?

Thanks!

dnanian
11-23-2005, 10:47 AM
We disable spotlight indexing while we copy, and restore it to its previous state on completion. We also preserve the spotlight "privacy" state automatically.

You definitely don't need to do what you're doing with v2.0... give it a try!

Hoosier_1701
11-23-2005, 02:19 PM
Fantastic! Thanks for the reply, Dave. Having to reboot was the only hassle in the entire process. I'm looking forward to my first 2.0 backup.

dnanian
11-23-2005, 02:30 PM
Great -- let us know what you think!

picaman
11-24-2005, 09:41 AM
Is this a separate issue from the tip described here (http://www.shirt-pocket.com/forums/showthread.php?t=361)? I have this script installed and it runs each time I backup.

Thanks for an fantastic product and a rock-solid new version that was free to registered users. Awesome.

:)

Jamie

dnanian
11-24-2005, 09:45 AM
Yep -- you don't need to run that script any more as long as you're running 10.4.3. Instead, turn indexing back on with "sudo mdutil -i on "/Volumes/the-backup-drive-name", and then drag it into your Spotlight preference pane's Privacy list.

We'll take it from there. :)

picaman
11-24-2005, 09:56 AM
You...you...made me use Terminal for the first time ;)

So I typed in "sudo mdutil -i on "/Volumes/Martha My Dear" which seemed to work fine as it just took the command and returned me to ">".

But then I tried to drag the disk image into the Spotlight privacy window, and it won't add it to the list. It's my internal disk which I use as a backup, but I boot off of an external disk.

Any ideas?

dnanian
11-24-2005, 10:02 AM
Try restarting, picaman, and see if it lets you add it then. If not, you'll need to remove a special folder at the top of the drive with:

sudo rm -r "/Volumes/Martha My Dear/.Spotlight-V100"

Be careful with that one, no mistakes. Then, restart and try again.

picaman
11-24-2005, 10:20 AM
The first restart didn't work, but the command and a subsequent restart solved the problem.

You rock. Thanks so much.

;)

Jamie

dnanian
11-24-2005, 10:48 AM
It is you, my friend, who rock. Party on. ;)

dirt farmer
11-26-2005, 12:48 AM
Hi Dave, just want to make sure I did things correctly. I, too, have never used Terminal before.

I, too, used the old script. After reading the instructions above, this is what I did:

1) In terminal, I typed ------> sudo mdutil -i on "/Volumes/backup".

I was asked for my password. A message appeared saying "indexing enabled on /Volumes/backup" . So far, so good.

2) I dragged my backup drive icon (image?) from the desktop to the spotlight privacy pane. It took it, no problems.

3) I launched superduper. In the Advanced options, my "Run shell script after copy completes" box was checked, with the old disable_spotlight script still there. I unchecked the box.

4) I trashed the disable_spotlight script from my Application folder.

Did I do everything correctly? Were steps 3 & 4 necessary?

Thanks for your help.

dnanian
11-26-2005, 12:50 AM
That's all you needed to do. 3 was necessary, 4 not, but it didn't hurt anything.

Should work fine!

Timmy
11-27-2005, 05:49 AM
We disable spotlight indexing while we copy, and restore it to its previous state on completion. We also preserve the spotlight "privacy" state automatically.



In a situation where we manually connect an external firewire drive and then perform a "Backup All files"/"Erase backup then copy" won't Spotlight start creating an index on the drive as soon as we turn it on the next time we connect the drive?

Oh, one other thing...
The fields for logging into this forum don't seem to be recognized by Safari for storage in the Keychain.
I know there is a 'remember me' option, but I assume that just sets a cookie.

dnanian
11-27-2005, 07:31 AM
Not if you put it in the Preferences tab, Timmy. We'll preserve the state, even across erase.

And, yes -- Remember me sets a cookie. I don't know why Safari wouldn't be working right here, as it does for me...

Timmy
11-29-2005, 05:40 AM
Not if you put it in the Preferences tab, Timmy. We'll preserve the state, even across erase.

Okay, cool.
So, I should add all of my backup volumes to the Spotlight privacy tab and then we won't have to worry about the drives being needlessly indexed.

But, what will be the Spotlight condition of a volume that is restored?
Will it have SL disabled?

I assume all of this pertains to 10.4.2 and not just 10.4.3, right?

Thanks.

dnanian
11-29-2005, 08:18 AM
Right. Backup volumes will be in the Privacy tab, and things should work fine in 10.4.3 and later.

Restored volumes will have their state maintained too, so they should be enabled (or whatever state they're in at the time of restore). If restored with Disk Utility it might end up in the Privacy panel -- just drag it out, and all will be well.

I don't think they fixed all the various privacy panel issues with 10.4.2, but it's possible they did. Give it a try -- if the drive stays in Privacy, even across a reboot or disconnect/reconnect, without SD! involved -- it will work.

Timmy
11-29-2005, 09:40 AM
Thanks Dave.

Timmy
12-01-2005, 04:26 PM
Spotlight still seems to be slowing down our backups under Tiger.

It is taking almost 40 minutes to fully clone a 13GB volume under Tiger, whereas cloning a larger volume on Panther takes less than 20 minutes.

Prior to updating to 10.4.3 I had added my backup volumes to the Spotlight privacy tab. Should I have added them again after updating to 10.4.3?? (which you had said fixes issues with the privacy tab...)

Or do backups just take longer on with Tiger?

dnanian
12-01-2005, 04:32 PM
You should ensure they're in there, yes -- are they?

Timmy
12-01-2005, 05:55 PM
You should ensure they're in there, yes -- are they?


They are listed in the Spotlight Privacy list.

dnanian
12-01-2005, 05:58 PM
So, we're preserving that state, and Spotlight *shouldn't* be indexing the drive. Interesting...

Timmy
12-01-2005, 06:29 PM
So, we're preserving that state, and Spotlight *shouldn't* be indexing the drive. Interesting...

Is there a way that I can confirm that Spotlight is in-fact indexing the drive while the backup is taking place?

Maybe some command or tell-tale sign that would specifically indicate that Spotlight is indexing the target as it is being written (other than the long backup time)?

Because, on 10.4.2 the backup would take longer than expected and then after it was completed there would be disk activity on the target disk (which I assume was Spotlight finishing its indexing)

Now with 10.4.3 the backup takes just as long, but there does not seem to be the disk activity after the clone is completed...

dnanian
12-01-2005, 06:33 PM
Well, I'd expect you to see the "dot" in the magnifying glass blinking, or something similar. What's weird is that we turn off indexing completely for the destination during the copy, and then we turn it back on again when we're done. It shouldn't be indexing at all.

Timmy
12-01-2005, 07:05 PM
Hmmm.
I will have to investigate this further...
Perhaps it's not a Spotlight issue after all...!

I will report back in this thread in a few days.

dnanian
12-01-2005, 10:09 PM
Hm. Please do: I'll be interested in what you find.

Timmy
12-02-2005, 11:55 AM
Hm. Please do: I'll be interested in what you find.

I'm still not sure what I'll be looking for, except a drastic speed boost in backup time, or for the Spotlight icon to be flashing during/after backup.

dnanian
12-02-2005, 12:05 PM
The flashing (just a dot in the magnifying glass) would typically indicate something's happening during the backup, though not necessarily the drive...

Do you do a permission repair before backing up (Options)? If so, try turning that off, too.

Timmy
12-09-2005, 05:44 PM
Well, I haven't been able to make a definitive determination that it is (or isn't) Spotlight that is causing our backups to take such a long time.

I will just use Smart Update from now on to make things a bit faster...

Is there something about Tiger that makes backups take longer than on 10.3.x?
Are there more small system files, or does the existence of meta-data, etc make the process inherently slower?

dnanian
12-09-2005, 05:59 PM
No, but Spotlight monitors every file access and can take time to do so. Try booting in Safe boot (hold down Shift during power on, and leave it down all the way to the desktop) to disable spotlight entirely -- that might tell you that it's causing your trouble.

george
12-21-2005, 07:25 PM
"Prior to updating to 10.4.3 I had added my backup volumes to the Spotlight privacy tab. Should I have added them again after updating to 10.4.3?? (which you had said fixes issues with the privacy tab...)"

slightly related issue...

I've run SD201 to backup/image OSX1039 volumes on a FW external that normally gets attached to an iBook running OSX143.

The partitions were NOT added to Spotlight's privacy Tab prior to backup. Should they have been?

I'm ready to Smart Update these volumes again, so what steps must been taken, if any, regarding this indexing issue before proceeding? (or, to ensure integrity of the backup set, must i erase/re-backup these volumes with them added to Privacy first?)

george

dnanian
12-21-2005, 09:20 PM
They don't have to be, no. But if you want them to not be indexed, you should add them to Privacy. It doesn't matter when you do it... just that you do if you don't want indexing.

Cuneyt
12-22-2005, 05:06 AM
It seems that I've missed this useful thread. I still use the old disable_spotlight script method on 10.4.3 and I'll do the steps described in this thread and won't use the script any more, as soon as possible.

Just before that, I suppose I should ask one thing. With the help of the disable_spotlight script when the backup disk mounted there is no indexing for backup drive, so far so good that's what I want it to be. But, if I startup from my backup disk, spotlight is starting to index (dot blinks). I first thought that was normal and Mac OS X re-enables the disabled indexing for the backup disk bacause it's the current startup disk. But I noticed that it also re-indexes my usual drive every time when I startup from my backup drive. Is this normal, will the new method prevent this from happening?

dnanian
12-22-2005, 09:35 AM
That's normal, and it happens because the files "change location", from "/" to "/Volumes/your drive/..."