CPU load averages

8 posts / 0 new
Last post
#1 Tue, 08/02/2011 - 15:08
marciano

CPU load averages

Hello,

I am worrying about high CPU load averages varying from 4 to 10 (dual core)

Not sure if related: with dmesg I am getting around 3 or 4 messages like this one (error 4 and error 6) php-cgi[27971]: segfault at 00000000010006f9 rip 000000386b87288e rsp 00007fff26e70e50 error 4

This is a clean Virtualmin installation on Centos 5.6 Linux 2.6.18-238.12.1.el5xen on x86_64

Number of visitors: 10000/day aprox Stable since fresh installation

I appreciate for any clue. Thanks. PS: coincident bots spidering my site (lot of pages) may be involved?

Tue, 08/02/2011 - 15:39
andreychek

Howdy,

Well, I've occasionally seen those PHP segfaults. How big a problem they are depends on how frequent they occur. If you're seeing 3-4 of those an hour, I'd say that's a problem. If you're seeing 3-4 a month, I'm not sure that I'd be too concerned about that.

Which PHP version are you using? If you're using the CentOS default of 5.1.6, you could try upgrading to 5.2.17 from the Virtualmin bleed repo.

However, I don't think that's related to your CPU load issue.

The key is to figure out what's generating the load, as well as verify that you have adequate resources.

One thing you may want to do is check your available RAM, by running:

free -m

That'll show how much RAM is available, as well as how much swap is being used.

Another thing would be to verify that you don't have a ton of messages in your mail queue. You can do that by running:

mailq | tail -1

I wouldn't be too concerned if it's lower than 50, maybe even 100. Much higher than that, and that's a bit on the unusual side.

Lastly, you can use the "top" command to get an idea of what programs are using a lot of the CPU.

-Eric

Tue, 08/02/2011 - 16:16
marciano

Hello Eric, thanks for your reply.

I have all programs up-to-date from virtualmin repositories. php is 5.1.6

I was bleeding in 2009 because of security holes in php 5.2. I don't see problem in RAM and CPU use.

RAM total used free shared buffers cached Mem: 2373 2192 180 0 60 1141 -/+ buffers/cache: 990 1382 Swap: 4094 299 3795

top - 17:12:30 up 27 days, 19:56, 1 user, load average: 3.34, 5.31, 5.31 Tasks: 157 total, 2 running, 155 sleeping, 0 stopped, 0 zombie Cpu(s): 46.8%us, 17.5%sy, 0.0%ni, 34.0%id, 1.0%wa, 0.0%hi, 0.2%si, 0.5%st Mem: 2429952k total, 2063976k used, 365976k free, 68576k buffers Swap: 4192944k total, 306308k used, 3886636k free, 1274912k cached

Mail queue is empty

What do you suggest? Thanks a lot

Tue, 08/02/2011 - 17:24
andreychek

Howdy,

It looks like you have enough RAM, and you aren't using that much swap.

What I would be doing is concentrating on what processes are running, and trying to determine from that where the trouble is.

When running "top", which processes repeatedly show up near the top of the output?

If you see a lot of "php" related processes, take note of what user owns the process. If you keep seeing one particular user's name over and over, you may need to review the apps running on that user's account. You may want to look at the log files for that domain, and see if anything appears out of the ordinary.

Similarly, you may want to look at Virtualmin's bandwidth information, in System Settings -> Bandwidth Monitoring. Take note of which domains are using a lot of bandwidth, then review those sites as well to see if one of them is part of the higher load you're seeing.

-Eric

Tue, 08/02/2011 - 19:43
andreychek

I fixed your earlier post.

As for your processes -- it looks like MySQL is getting hit pretty hard... which may mean you're just getting a lot of traffic, or it could mean that an app you're using is hard on the database.

I also see that there's a process called "searchd" which is using quite a bit of CPU time. That could be related so the problems you're seeing.

-Eric

Wed, 08/03/2011 - 19:23
marciano

Hello Eric, I am looking into mysql queries. mysql_tuner does not show big problems. I've enabled log long queries with log-slow-queries=/var/log/slow_query_log but few data was recorded (2secs) Show processlist only displays a few of queries each refresh, time=0 or so. Phpmyadmin shows more info. An average of 4000 queries/sec seems a little high. I am looking into it, I don't see anything wrong, lot of cached queries Max concurrent connections: 13

Just now I had to restart Apache because of [warn] mod_fcgid: can't apply process slot for ..(any http request) I never got this issue. Traffic and other related stuff are similar to months ago, less than in May, for example.

searchd is a daemon from a search engine. It does not use mysql resources. Thanks

Fri, 08/05/2011 - 15:39
marciano

I've changed some code and got drastically reduce cpu use by mysqld. My next question is if is it normal to have high php-cgi cpu values Thank you

top - 16:36:24 up 2 days, 8:03, 2 users, load average: 2.08, 2.65, 2.87 Tasks: 170 total, 2 running, 168 sleeping, 0 stopped, 0 zombie Cpu(s): 52.0%us, 15.0%sy, 0.0%ni, 29.3%id, 1.2%wa, 0.0%hi, 0.5%si, 2.0%st Mem: 2429952k total, 2226280k used, 203672k free, 72940k buffers Swap: 4192944k total, 286192k used, 3906752k free, 1265160k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23658 apache 16 0 214m 49m 7440 S 31.0 2.1 0:57.65 php-cgi
1947 apache 16 0 196m 31m 7436 S 26.3 1.3 0:14.40 php-cgi
11386 apache 17 0 205m 41m 7440 R 16.0 1.7 0:42.32 php-cgi
6034 root 25 0 66420 908 740 S 6.0 0.0 74:08.65 searchd
9852 mysql 15 0 345m 76m 4576 S 5.3 3.2 115:25.58 mysqld
11331 apache 16 0 203m 39m 7440 S 2.7 1.7 0:43.09 php-cgi
6530 virginia 15 0 140m 12m 1736 S 1.3 0.5 0:23.21 proftpd
1612 apache 15 0 346m 4560 1896 S 1.0 0.2 0:00.54 httpd
4091 apache 15 0 346m 4532 1896 S 1.0 0.2 0:00.26 httpd
1348 apache 15 0 346m 4620 1896 S 0.7 0.2 0:00.42 httpd
3360 apache 15 0 346m 4772 1920 S 0.7 0.2 0:00.23 httpd
6168 apache 15 0 346m 4540 1888 S 0.7 0.2 0:00.13 httpd
6217 apache 15 0 346m 4592 1892 S 0.7 0.2 0:00.12 httpd
7267 apache 15 0 346m 4508 1876 S 0.7 0.2 0:00.02 httpd
11494 apache 15 0 346m 4592 1896 S 0.7 0.2 0:01.23 httpd
27057 apache 15 0 346m 4580 1892 S 0.7 0.2 0:00.76 httpd
295 root 10 -5 0 0 0 S 0.3 0.0 2:00.02 kjournald
6216 apache 15 0 346m 4780 1920 S 0.3 0.2 0:00.17 httpd
10105 apache 15 0 346m 4652 1892 S 0.3 0.2 0:01.18 httpd
27371 apache 15 0 346m 4568 1900 S 0.3 0.2 0:00.56 httpd
28130 apache 15 0 346m 4564 1892 S 0.3 0.2 0:00.56 httpd
30843 apache 15 0 346m 4612 1892 S 0.3 0.2 0:00.42 httpd
32480 apache 15 0 346m 4652 1920 S 0.3 0.2 0:00.42 httpd

Fri, 08/05/2011 - 23:25
andreychek

I've changed some code and got drastically reduce cpu use by mysqld. My next question is if is it normal to have high php-cgi cpu values Thank you

Well, whether that's normal depends on the application... some applications require more CPU than others :-)

Also, if you're using FCGID, that means the php-cgi processes may stick around for more than just one session, as they cache code in memory in order to speed up site access. In that case, the CPU time you see is over the course of that process serving multiple users.

-Eric

Topic locked