XPF3 Synchronization Mayhem |
September, 22, 2003 6:48 PM |
chibi_delenn |
Well, this is something that I hope nobody else here has experienced. Today I decided to use XPF3 to boot my WD 200 GB HD's OS X partition (needs the Helper option for that). Well, after many an attempt (see note below after this rant/bug report) to boot the drive's OS, I finally got in to my gloriously fast(er) 10.2.6. The very first thing I saw, even before the USB Overdrive shareware reminder came up was "XPostFacto Synchronization in Progress..." notice box. I figured, OK, whatever, even though I had JUST restarted and done nothing yet, it shouldn't need to sync so early on, but I let it go. Then I opened Apple System Profiler. It unexpectedly quit (does that on occasion). Upon that app quitting, XPF's sync box came up AGAIN. This time I began to worry. But it was faster than the first time, so I figured "ok, I'll let it go again....this time." Big mistake. I then opened up System Preferences so I could activate software update. I found the 10.2.8 update (along with Java 1.4.1 which I failed to install last time), and selected both to install. Guess what happened next? Yep, you guessed it. XPF's sync came up AGAIN, but this time it was DURING THE GODDAMNED 10.2.8 INSTALL. Now I was furious. Each and every time XPF's sync box comes up, my system slows to a crawl. It's as if my booted OS's files are being monitored *constantly*. Every time I open and close an app, that damned synchronization box shows up. And for it to show up *during* a software update was the final straw. XPF 3 in OS X is akin to XPF2.2.5 in OS 9, i.e. either won't launch, launches with a menubar and nothing else, launches with menubar + OWC splash screen and doesn't let me DO anything, or randomly, if it feels like it, shows the proper list/settings window (It took me 37 launches of XPF3 to get that settings window a la OS 9). I found no way to turn off synchronization, which means I now have to worry about XPF itself causing system problems. This is not a good thing. I'm not sure if XPF3 updates the Helper disk files to match the OS files, or if it changes the OS system files to match the Helper files, but if it's the latter, and it's doing this while a software update install is in progress, then there is a MAJOR PROBLEM THAT NEEDS TO BE ADDRESSED. Now, regarding my many attempts to even boot said 10.2.6 partition: There have probably been many of you that have gotten the oh so fun "stop sign" icon upon attempting to boot OS X using XPF3, especially when using its Helper function. XPF3's files are apparently even more sensitive with regard to BootX than before. Unfortunately, XPF3's "Reinstall BootX" function seems to have been somewhat neutered, probably inadvertantly, as it does not work, or at least, wouldn't work for me. What I ended up doing was launching XPF 2.2.5, selecting my target startup disk, reinstalling BootX, and holding option down on restart, THEN using XPF 3 to start that partition up w/ a helper disk. Frustrating, annoying, but the workaround works. KEEP THAT OLD XPOSTFACTO AROUND JUST IN CASE. - Chibi Delennâ„¢ |
. |
RE: XPF3 Synchronization Mayhem |
September, 23, 2003 8:57 AM |
OSXGuru |
. |
No problem, Chibi -- you've been giving me some very useful information and impressions. The whole user experience around synchronization was my biggest concern about XPF 3, and it needs to be done right and reliably if FireWire booting is going to be a real option. So it's good for me to hear what you have to say. What I realized reading your comments is that the current UI for synchronization is not confidence-insipring. That is, even if I could fix the technical problems in the current approach (which I probably could with enough effort), the user experience would still be flawed. Basically, to have a window pop up at unexpected moments telling you that momentous things are happening that you have no control over is bad user experience. So, at the very least, I'm going to give the other approach a try--I think it might be a much better user experience. |
. |
RE: XPF3 Synchronization Mayhem |
September, 23, 2003 2:44 AM |
chibi_delenn |
. |
jonck, The problem in your suggestion is that XPF would still have to poll for changed files, since that is the only way it'd know of the update happening (that I know of) before restarting. And it generally either isn't possible, or is highly discouraged to force quit the installer in OS X. I've seen things get a bit munged from doing that, which is what XPF would have to do in order to forego a shutdown while it copies. Forcing me to boot back to 9 is something I'd be a bit angry with as well. While I do boot back to 9 fairly often (games bite in OS X on my mac), I should be able to have the option of manually updating from within OS X, and not in OS 9. Once the installer has finished optimizing, THEN you can safely (usually) launch XPF to manually synchronize your files, as this only utilizes file copying, and not any specialty services that are updated in most OS updates. This is acceptable. Hopefully Ryan's ultimate solution will bring peace to us all with XPF3. Of course, I too, would much rather see Panther (and maybe dual L2/L3 cache support) in XPF3, but that will wait...gotta get the app itself working right first. :) - Chibi Delennâ„¢ |
. |
RE: XPF3 Synchronization Mayhem |
September, 23, 2003 1:55 AM |
jonck |
. |
Hi Ryan, Just a thought, if you can forego polling by requiring that the user boot back into OS 9 after an update, then why not just make that a requirement of XPF? If you say so very explicitely in the XPF readme, then people should follow this guideline. What I'm trying to say here is that the people using XPF are not the average mac user who feel that everything should work right out of the box, since if they felt that, they wouldnt (or shouldn't) be tinkering so much with their old macs to get OS X running on it. If you can bypass the problems you are having here with a simple requirement that people boot back into 9 once after running an upgrade of OS X, then IMHO you should not spend any more of your time on trying to solve this problem. I for one would much rather see you spend your valuable time on getting Panther to run on our ancient machines :-) All the best, Jonck |
. |
RE: XPF3 Synchronization Mayhem |
September, 23, 2003 1:50 AM |
chibi_delenn |
. |
Before I forget... If you ever feel that I'm out of line, feel free to yell right back at me Ryan. I may have only one ear and one eye (I'm dead serious about that), but I do listen, and I listen well. If you have any suggestions or workarounds for me to try, I'll gladly take those as well. :) Just thought I'd let you know I understand that I too can be...irritating. Anyway, try to keep cool (be glad you aren't in NoCal right now - 100 deg avg temps here, 85 deg right now at midnight, ugh), I'm sure you'll make XPF3 a winner. - Chibi "Gimp" Delennâ„¢ |
. |
RE: XPF3 Synchronization Mayhem |
September, 23, 2003 12:34 AM |
OSXGuru |
. |
I'll respond to your points in turn: 1. The polling itself is reasonably inexpensive. If you run ps, for instance, you'll see that it's using very, very little time. Of course, the actual copying takes some time, but there's nothing I can do about that. Actually, I suppose that is a little too glib. Part of the problem with the way I have structured this is that the user has no control over when the copying takes place. That is, the overall user experience is bad in ways that aren't entirely necessary. 2. If you look at the source for xpfbootsupportd, what it's trying to do is notice when a file is being written to, and then wait for it to be finished before doing the copy. This appears to work well for the mach_kernel and Extensions.mkext, but it is problematic for the Extensions folder itself. I think I understand why, and I've opened a bug for that as well: Fix race condition when copying extensions folder 3. The window that pops up isn't a great user interface, admittedly. It was supposed to only pop up during an install, which I figured was not a time when people would be doing a lot of productive work. But, that may not be a very good assumption. Of course, polling is bad, but it can't really be avoided entirely here. You're quite right that rebooting into Mac OS 9 and then using XPF would solve the synchronization issues. But that isn't the usual scenario. The usual scenario is that you run Software Update, your kernel extensions get updated, and you reboot right back into Mac OS X. So, without some kind of synchronization with the helper, you'll have a mismatched set of kernel extensions loaded (the old ones from the helper, and the new ones that were just installed). Given both the technical problems and user experience problems with the current synchronization method, I am seriously thinking of switching to the other available option, which would be to launch an application with a UI when synchronization is required. The application can then perform the sync when it quits, which should work out pretty well, because it will normally quit at logout (i.e. shutdown). And a UI application can delay logout as long as it needs to. So the overall user experience ought to be a lot friendlier. Basically, you've convinced me that the way I've got it set up now is bad user experience. |
. |
RE: XPF3 Synchronization Mayhem |
September, 22, 2003 11:17 PM |
chibi_delenn |
. |
Ryan, The problem with constantly polling for changed files is twofold. 1) It slows the system down immensely at times, and even when it's overwith fairly "quickly", it's an unexpected and very annoying interruption. OS X changes its files much to inconsistently under normal use for this kind of polling to be effective without either aggravating the user, or in my case, royally screwing my system up (see #2). 2) XPF polled during software update. This is BAD. Very Badâ„¢. Why? It managed to see a changed file, and it copied that file, but the installer had not finished writing the file to begin with! The installer apparently did not like that, and though it put up no warning stating that an unauthorized access to the system was made, sure enough, on reboot, it froze during loading the .kexts. 3) OK, I thought of the third problem. That insidious window that pops up every time this thing start sychronizing. There is *no* way to dismiss it, or to get at anything *behind* it. Period. I know you want to make sure the user knows not to shutdown while XPF is doing its work, but let's say the user has a timed series of events/inputs that are needed to be input and XPF rears its ugly head. The user is stuck, and can't do anything, and if it's a large sequence driven project, might have to start all over again because he/she couldn't input anything while XPF's window was in place. I honestly believe that polling is a relatively bad practice. Many apps that made use of external hardware in the past polled USB and SCSI chains constantly, and if either a device was turned off in the chain, or just because of the polling, the computer would grind to a halt (Toast is notorious for this with its CD Reader extension in OS 9). I believe that most (note I do say most - not everyone here reads/speaks the best Engrish, er English) of the users here that manage to use XPF could remember to manually sync in OS X before shutting down. If not, and even at that, XPF does sync before restarting when in OS 9, right? So theoretically, unless you never restart to OS 9 again (in which case a scheduler app would be more apropos), polling should be for the most part unnecessary in OS X. Please remember also that we're on slow machines. We aren't using UberMacs like MDDs and G5s where polling wouldn't make even the weakest CPU flinch. I understand the frustration you must be going through right now trying to please us all at once. We do sincerely appreciate your efforts (just take a look at the Bugzilla reports and comments from users here). But this one feature of XPF3 can be a potentially fatal one, as it was in my case when it activated itself *during* a software update install. Any alternatives you think of, I'm willing to try out (I did make a backup clone, which I had restored while I was away running errands - copying 470,000 files takes a while). One more note - you might want to have XPF do a search for "SonnetCache.kext" and any Powerlogix/XLR8 related items in the System folder before it installs the L2CacheStart items. That was causing me a bit of a headache wondering what happened to my L2 cache all of the sudden after bootup. I use the SonnetCache.kext because it (supposedly at least - I have no method of checking the actual speed unless I install another 3rd party .kext set, ugh) enables the cache at the proper ratio. I'm lucky I keep cloned backup partitions. It keeps me from having to rebuild from scratch all the time. :) Powderhaus: XPF's sync feature is already a "background app", and if you see what I mentioned above, it can be a painful slowdown inducing one at that. Background apps aren't anything near ideal, nor wanted actually on slower machines if they can be avoided. Assuming XPF3 syncs itself while in OS 9 (during the Restart process when using a Helper), most users will never have to sync inside OS X at all. Sadly, there doesn't seem to be an "ideal" solution just as of yet. But I'm sure somebody will come up with something....soon, I hope. :) - Chibi Delennâ„¢ |
. |
RE: XPF3 Synchronization Mayhem |
September, 22, 2003 9:15 PM |
powderhaus |
. |
I just put this on bugzilla ------ Additional Comment #3 From Jim Smith (Powderhaus) 2003-09- 22 19:14 ------- Would there be anyway to have a background application (the will always refuse to shutdown, but when it gets the shutdown signal have it check the files for changes. If there is nothing wrong it should be able to exit in time for normal shut down, if there is a change you have have the program send the shutdown signal again, or if that is not possible you can have a continue with shutdown/reboot button. |
. |
RE: XPF3 Synchronization Mayhem |
September, 22, 2003 8:38 PM |
OSXGuru |
. |
The purpose of the synchronization process is to copy files from the root volume to the helper volume. The synchronization changes the files on the helper volume--it doesn't change files on the root volume. The idea is to make sure, when performing upgrades (like the the upgrade to 10.2.8), that the files on the helper stay in sync with the new files on the root volume. Since the Mac OS X installer has no way of knowing about the helper, it doesn't copy files there by itself. The synchronization process is continuously polling for changes to the /mach_kernel file, the Extensions.mkext file, and the /System/Library/Extensions folder. The problem with doing it at shudown is that there is no reliable way to be notified of shutdown in such a way that you are guaranteed to have enough time to do your work. There are some alternatives that I am thinking about, though--here is the bug: Test alternatives to polling So the "normal" part of what you describe is the synchronization process that occured when you updated to 10.2.8. Of course, the synchronization process should not start every time you open an application! I've opened a bug for that: Synchronization process runs whenever an application is launched One question I have is whether it is always the case that it goes away very quickly in those cases. Also, it would be interesting to look at your system.log for those times-- xpfbootsupportd writes a message to the system log saying what file it thought had changed. The problem launching XPostFacto in Mac OS X is probably related to some error condition that occurs before the UI is up. Looking at the log file might provide a clue--it should be at ~/Library/Preferences/XPostFacto Log. I've opened a bug for this as well: Mac OS X version not launching properly I'll have to test the Reinstall BootX command and see if it seems to be working. I thought it was working, but I guess we'll see. Here's the bug: Test "Reinstall BootX" command |
. |
RE: XPF3 Synchronization Mayhem |
September, 22, 2003 7:05 PM |
chibi_delenn |
. |
Extra note to Ryan (since I can't edit posts - BG crappy forum software): This is not me yelling at you. It's me being frustrated at losing bootability on that partition due to this happening. If you know why/how XPF3 does this, please let me know. If there's a way to "schedule" syncrhonizations, (it did not do so at shutdown, which is when it *should* do it), I would love to know that as well. Thank you for your time and patience! - Chibi Delennâ„¢ P.S. - If anybody still wonders why I make cloned backups of my working OS X partitions, this is a *DAMN* good reason why. |
|
|