AFP eating up CPU
|
View:
New views
9 Messages
—
Rating Filter:
Alert me
|
 |
AFP eating up CPU
by Dan Rader
::
Rate this Message:
I was experiencing this problem as well. I found a DNS error and fixed it
and then restarted the server. This has taken care of my CPU maxing out problem until yesterday. I got a call around noon that weird things were happening to my logged in users. There docs were disappearing and wallpaper pictures were gone, etc. The secretary in that building told everybody to shut down their computers until further notice. Since I didn't actually see this weirdness I told a couple of users to turn their computers back on and log in. It appears as though they had lost there doc prefs, sidebar prefs, and background prefs. These were the things that were immediately noticeable. I found out later that most of them have Firefox profile issues as well. This morning I noticed that my CPU % has been over 50% overnight when nobody is supposed to be here and shouldn't have anything going on and this morning it has been averaging around 75% and looks like it is slowly creeping up. I don't want to have to restart the server again. My users are getting frustrated. Has anyone found any solutions? _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by Dan Shoop
::
Rate this Message:
On Oct 14, 2008, at 11:47 AM, Dan Rader wrote:
You presume that there's a single cause to this. Unfortunately it's not that simple. Your problem is likely different from others. Given dtrace and sc_usage you can look deeper. Remember though that %CPU is not a metric, and if a process is consuming most all the CPU that this implies there's no other processes that are competing for CPU significantly. What normally you find is that when processes are using "excessive" cpu that they're spending a large amount of system time, which indicates something fundamental wrong, like latency issues, bad DNS, misconfiguration etc. It's spending time there b/c it's made system calls that aren't getting serviced. Find out why and you solve your mystery. Other issues occur when you've got waits that never return. That indicates, again, something fundamental that's out of line or problematic. Guess my overall suggestion is that the issue isn't really with AFP but something in the system that's not right. -d ------------------------------------------------------------------------ Dan Shoop Computer Scientist iWiring / U.S. Technical Services AOL IM .................... iWiring Nextel .................... 1-714-363-1174 Operations TOC (24/7) ..... 1-866-901-USTS USTS Offices .............. 1-714-374-6300 at the USTS Tactical Operations Center (above) who can reach me by radio. _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
|
|
|
Re: AFP eating up CPU
by Anders Pihl
::
Rate this Message:
I had this problem and in my case it was the clients time settings
that was the problem. In a lot of clients the clock was wrong. I changed the time settings so all computers had the same time as the server and now the CPU is normal again. Anders 23 okt 2008 kl. 19.54 skrev Jeremy Wellner: > I'll second that notion. > > We've been having some problems with our core home directory servers > and high AFP CPU and I was finally able to bring them offline last > night for a bit of maintenance. > > Of the 4 I worked on, I believe all of them (it was getting late and > cold medicine is awesome...) had volume header block, incorrect > #ACLs and to add icing an incorrect # of extent attributes. > > They took FOREVER to run disk utility, however 3 of the 4 are quite > happy today. The fourth one took so long running DU that I just let > it run overnight and had it up the next morning. > > Also part of the maintenance I have a script I churn thru to clean > out .DS_Store files, user home caches and a few other things. > (Didn't have time to run the script and let it finish hence some of > the issues still). > > Not to ask a silly question... but how do you peek into the process > with dtrace and sc_usage? > > Jeremy Wellner > Technology Group > Stanwood-Camano School District > jwellner@... > > Internal: x3505 > External: 360-629-1205 > Cell: 425-418-4972 > > "Time travel?...The evidence is all around us. I'm talking about how > every time we make an insurance claim we discover that somehow > mysteriously the exact thing we're claiming for is now precisely > excluded from our policy." > - Salmon of Doubt, Douglas Adams (1952-2001) > > > > > On Oct 23, 2008, at 9:52 AM, Dan Shoop wrote: > >> >> On Oct 14, 2008, at 11:47 AM, Dan Rader wrote: >> >> >> >> I was experiencing this problem as well. I found a DNS error and >> fixed it >> and then restarted the server. This has taken care of my CPU >> maxing out >> problem until yesterday. >> >> I got a call around noon that weird things were happening to my >> logged in >> users. There docs were disappearing and wallpaper pictures were >> gone, etc. >> The secretary in that building told everybody to shut down their >> computers until further notice. Since I didn't actually see this >> weirdness I told a couple of users to turn their computers back on >> and log >> in. It appears as though they had lost there doc prefs, sidebar >> prefs, >> and background prefs. These were the things that were immediately >> noticeable. I found out later that most of them have Firefox profile >> issues as well. >> >> This morning I noticed that my CPU % has been over 50% overnight >> when nobody is supposed to be here and shouldn't have anything >> going on >> and this morning it has been averaging around 75% and looks like it >> is >> slowly creeping up. >> >> I don't want to have to restart the server again. My users are >> getting >> frustrated. Has anyone found any solutions? >> >> >> >> >> You presume that there's a single cause to this. Unfortunately it's >> not >> that simple. Your problem is likely different from others. >> >> >> Given dtrace and sc_usage you can look deeper. >> >> >> Remember though that %CPU is not a metric, and if a process is >> consuming >> most all the CPU that this implies there's no other processes that >> are >> competing for CPU significantly. >> >> >> What normally you find is that when processes are using "excessive" >> cpu >> that they're spending a large amount of system time, which indicates >> something fundamental wrong, like latency issues, bad DNS, >> misconfiguration etc. It's spending time there b/c it's made system >> calls >> that aren't getting serviced. Find out why and you solve your >> mystery. >> Other issues occur when you've got waits that never return. That >> indicates, again, something fundamental that's out of line or >> problematic. >> >> >> Guess my overall suggestion is that the issue isn't really with AFP >> but >> something in the system that's not right. >> >> -d >> >> >> ------------------------------------------------------------------------ >> Dan Shoop >> Computer Scientist >> iWiring / U.S. Technical Services >> >> >> [ mailto:shoop@... ]shoop@... >> AOL IM .................... iWiring >> Nextel .................... 1-714-363-1174 >> Operations TOC (24/7) ..... 1-866-901-USTS >> USTS Offices .............. 1-714-374-6300 >> >> >> For immediate response for urgent matters please speak to the Duty >> Officer >> at the USTS Tactical Operations Center (above) who can reach me by >> radio. >> >> >> >> _______________________________________________ >> Do not post admin requests to the list. They will be ignored. >> Macos-x-server mailing list (Macos-x-server@...) >> Help/Unsubscribe/Update your Subscription: >> http://lists.apple.com/mailman/options/macos-x-server/jwellner%40stanwood.wednet.edu >> >> This email sent to jwellner@... >> >> <Attach0.html> > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Macos-x-server mailing list (Macos-x-server@...) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/macos-x-server/agp%40rudebecks.se > > This email sent to agp@... > Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by David Haines-2
::
Rate this Message:
>> On Oct 23, 2008, at 9:52 AM, Dan Shoop wrote: >> >> You presume that there's a single cause to this. Unfortunately it's >> not >> that simple. Your problem is likely different from others. >> >> >> Given dtrace and sc_usage you can look deeper. >> >> >> Remember though that %CPU is not a metric, and if a process is >> consuming >> most all the CPU that this implies there's no other processes that >> are >> competing for CPU significantly. >> >> >> What normally you find is that when processes are using "excessive" >> cpu >> that they're spending a large amount of system time, which indicates >> something fundamental wrong, like latency issues, bad DNS, >> misconfiguration etc. It's spending time there b/c it's made system >> calls >> that aren't getting serviced. Find out why and you solve your >> mystery. >> Other issues occur when you've got waits that never return. That >> indicates, again, something fundamental that's out of line or >> problematic. >> >> >> Guess my overall suggestion is that the issue isn't really with AFP >> but >> something in the system that's not right. On Oct 23, 2008, at 1:54 PM, Jeremy Wellner wrote: > > > Not to ask a silly question... but how do you peek into the process > with dtrace and sc_usage? man sc_usage man fs_usage For DTrace, there are a number of examples to look through in /usr/ share/examples/DTTk http://www.mactech.com/articles/mactech/Vol.23/23.11/ExploringLeopardwithDTrace/index.html http://www.samag.com/documents/s=9915/sam0512a/0512a.htm I've yet to do a significant amount with DTrace (not having moved much past iosnoop, opensnoop & rwsnoop), but have a look at man filebyproc.d iosnoop , rwsnoop, syscallbyproc.d _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by Nathan Zamprogno
::
Rate this Message:
We are also experiencing chronic issues with AFP CPU usage spiking
and slowing dramatically (without actually "crashing") as a reboot, sometimes twice a day is necessary to keep our labs ticking over (OS-X Server 10.5.5). One contributory (but not sole) cause is AFP logging. Many admins will instinctively increase the logging level to try and diagnose an issue, but if you go into Server Admin->AFP make sure you aren't logging every last thing like all file opens and creations. I restrict logging to Login/Logout and "create folder" and it helped a little. AFP is still a gaping reliability hole in Apple's server product. I've been waiting for them to fix it for about 5 years. At 10:47 AM -0500 14/10/08, Dan Rader wrote: >I was experiencing this problem as well. I found a DNS error and fixed it >and then restarted the server. This has taken care of my CPU maxing out >problem until yesterday. > >I got a call around noon that weird things were happening to my logged in >users. There docs were disappearing and wallpaper pictures were gone, etc. > The secretary in that building told everybody to shut down their >computers until further notice. Since I didn't actually see this >weirdness I told a couple of users to turn their computers back on and log >in. It appears as though they had lost there doc prefs, sidebar prefs, >and background prefs. These were the things that were immediately >noticeable. I found out later that most of them have Firefox profile >issues as well. > >This morning I noticed that my CPU % has been over 50% overnight >when nobody is supposed to be here and shouldn't have anything going on >and this morning it has been averaging around 75% and looks like it is >slowly creeping up. > >I don't want to have to restart the server again. My users are getting >frustrated. Has anyone found any solutions? > > > _______________________________________________ >Do not post admin requests to the list. They will be ignored. >Macos-x-server mailing list (Macos-x-server@...) >Help/Unsubscribe/Update your Subscription: >http://lists.apple.com/mailman/options/macos-x-server/nathan%40wycliffe.nsw.edu.au > >This email sent to nathan@... -- Nathan Zamprogno, IT Manager, Wycliffe Christian School. nathan@... Ph: 0247 536 422 ext 911 Mob: 0412 141 811 http://www.wycliffe.nsw.edu.au "I can tolerate being judged; far better than to be of no consequence." -Mr Spock, "World Enough and Time" _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by Stewart Lawler
::
Rate this Message:
On 23/10/2008, at 9:21 AM, Dan Shoop wrote: > You presume that there's a single cause to this. Unfortunately it's > not that simple. Your problem is likely different from others. > > Given dtrace and sc_usage you can look deeper. > > Remember though that %CPU is not a metric, and if a process is > consuming most all the CPU that this implies there's no other > processes that are competing for CPU significantly. > > What normally you find is that when processes are using "excessive" > cpu that they're spending a large amount of system time, which > indicates something fundamental wrong, like latency issues, bad DNS, > misconfiguration etc. It's spending time there b/c it's made system > calls that aren't getting serviced. Find out why and you solve your > mystery. Other issues occur when you've got waits that never return. > That indicates, again, something fundamental that's out of line or > problematic. ..to follow up on this, I have a machine (dual quad-core intel xserve, 10.5.4) that's been sitting at 50% CPU load for the last few weeks no matter how many (or few) actual connections there are. (this machine replaced a dual G5 who's AFP was consistently using 150% cpu so i guess it's an improvement) so on Dan's hint i ran sc_usage against the AFP process and, although it's mostly all greek to me, it did seem that there were a lot of calls to swtch_pri going on. (I should've kept a grab of the figures, but the number against swtch_pri were massive compared to the other calls.) where it got interesting was when i decided to restart AFP from serveradmin. in a nutshell, it wouldn't. I couldn't kill -9 it either.. it was a zombie. But looking at sc_usage again there was swtch_pri all by itself still raking up the numbers... and with a pretty guilty look on it's face if i do say so. So i rebooted the box and now the cpu graph is at a normal level, and there's no mention of swtch_pri cropping up at all in sc_usage. Bit of googling led me to http://www.gnu.org/software/hurd/gnumach-doc/Hand_002dOff-Scheduling.html which describes swtch_pri thusly: > The system trap swtch_pri attempts to switch the current thread off > the processor as swtch does, but depressing the priority of the > thread to the minimum possible value during the time. Sounds great. Now, would anyone have any idea how i can use this information to fix my server.. or is it bug report time? ..S. _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by John C. Welch
::
Rate this Message:
On 11/10/08 6:05 PM, "Stewart Lawler" <s.lawler@...> wrote:
> so on Dan's hint i ran sc_usage against the AFP process and, although > it's mostly all greek to me, it did seem that there were a lot of > calls to swtch_pri going on. (I should've kept a grab of the figures, > but the number against swtch_pri were massive compared to the other > calls.) Try running fs_usage too, only dump the output to a text file. It logs all file system access. -- John C. Welch Writer/Analyst Bynkii.com Mac and other opinions jwelch@... _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
|
|
Re: AFP eating up CPU
by Dan Shoop
::
Rate this Message:
On Nov 10, 2008, at 6:05 PM, Stewart Lawler wrote:
There's no doubt about it, AFP Server is a big user of CPU time, but not all CPU time is equal. One thing that AFP seems to use a lot of CPU time for is "hurrying up and waiting".
If you examine the thread and the stack in dtrace or Instruments you can see how this relates. swtch_pri is a Mach trap.
A real "Z"? Were you still watching it with sc_usage when you tried to first kill it?
Sounds like you may be in a wait for loop and you're schedulable and not doing much much in terms of CPU, you keep asking to be switched out. Check your context switches. The scheduler is picking you up and you're saying reschedule me. The case of the 50% CPU then is indicative then of a behavior that indicates that there's probably very little else schedulable as a candidate for other CPU on your system. The fact that this accounts for 50% of your CPU is then moot since your server has little else to be doing. You could think of such swtch_pri time as being an "idle process". So the real question is: What performance impact are you seeing? Are you I/O bound? Have you examined your storage architecture? What is your system load? What is non-AFP system load?
And when your system restarted how did it behave with respect to call usage and performance? And no, it's not bug report time. -d ------------------------------------------------------------------------ Dan Shoop Computer Scientist iWiring / U.S. Technical Services AOL IM .................... iWiring Nextel .................... 1-714-363-1174 Operations TOC (24/7) ..... 1-866-901-USTS USTS Offices .............. 1-714-374-6300 at the USTS Tactical Operations Center (above) who can reach me by radio. _______________________________________________ Do not post admin requests to the list. They will be ignored. Macos-x-server mailing list (Macos-x-server@...) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/macos-x-server/lists%40nabble.com This email sent to lists@... |
| Free embeddable forum powered by Nabble | Forum Help |
