Well 2500 users does not seem like many more than normal, but pages are not very quick right now... Notice generation times about 1 second.
Server Overload
- Thread starter mike123abc
- Start date
- Latest activity Latest activity:
- Replies 28
- Views 3K
- Status
Is the site still slow for you?
I have been on for sometime now and it was never slow for me "Page generated in 0.21448".
I have been on for sometime now and it was never slow for me "Page generated in 0.21448".
This site is by far the slowest site I access on a regular basis and has been for a long time.
While some days are worse than others, SatGuys always takes at least 2 to 3 times longer to load and change pages as any other site I access.
While I'm not saying it doesn't have something to do with my setup, I'm at a loss as to what else I could try. I've done the usual like clearing cookies (logging off), clearing my cache, deleting all temporary internet files, run PCPitstop's Optimize everyday, reset and even reinstalled IE7 and now am using FireFox.
At any given time, I can go to DBSTalk, DBSForums, AVSFourms and browse 2 or 3 times faster. It's frustrating because I'd much rather be here. This isn't the first time I've posted this either but never got much for help other than it must be my system or my ISP. Yet this, I repeat, this is the only site I consistently have these problems.
While some days are worse than others, SatGuys always takes at least 2 to 3 times longer to load and change pages as any other site I access.
While I'm not saying it doesn't have something to do with my setup, I'm at a loss as to what else I could try. I've done the usual like clearing cookies (logging off), clearing my cache, deleting all temporary internet files, run PCPitstop's Optimize everyday, reset and even reinstalled IE7 and now am using FireFox.
At any given time, I can go to DBSTalk, DBSForums, AVSFourms and browse 2 or 3 times faster. It's frustrating because I'd much rather be here. This isn't the first time I've posted this either but never got much for help other than it must be my system or my ISP. Yet this, I repeat, this is the only site I consistently have these problems.
I am surprised, as the site has been very zippy for me over the past month. There was a time when it was quite sluggish, but I am not having any problems.
Have you run tracert to see if there are any bottlenecks in your connection, getting from Vermont to Texas, where the server is physically located?
Have you run tracert to see if there are any bottlenecks in your connection, getting from Vermont to Texas, where the server is physically located?
Most of the time it goes pretty fast. Every now and then it slows for a bit. It is much better than when the last big upgrade first happened. Perhaps a ton of people check the site at the same time. Something causes page generation times to spike every now and then. When they go up to a second is when the site feels really slow. Then after a while it corrects itself and gets zippy again. Right now it seems fast page generation on this page is .14501
This last software update seems to have had a few more bugs than usual. BTW being 100 miles from the server gives me 17ms ping time, only changes from ATT to Level3 in FT Worth.
This last software update seems to have had a few more bugs than usual. BTW being 100 miles from the server gives me 17ms ping time, only changes from ATT to Level3 in FT Worth.
Ok here is what's up.
95% of the time there are no issues, the server functions like it should. Speeds are overall very good to excellent.
5% of the time though there is a slowdown of the system which I have not put my finger on yet.
We run some software on our server called Cpanel, which basically controls the server and helps manage the server. We were using Cpanel 10 and our server loads were below 1.0 (normally around 0.70) then Cpanel 11 came out and something happened there which caused the loads on the server to shoot up and stay around 7.50 or higher. (A lot of folks were complaignin the server was slow... and it was slow because of this reason)
I did a bunch of tweeking when this happend and got the server load down to between 2.1 and 4.1. The speeds picked back up again to an acceptable level.
From where I sit, our server is a powerhouse, we only host SatelliteGuys and DishRetailer on in (Dish retailer is a small forum that normally has 20 - 30 people online.) because of this and the specs with our server I know that something is still not 100% on it. A server like this sould be able to handle 20 or more busy sites like ours without issues.
But it don't, and thats a problem.
There are two options I can do...
1) There is a newer version of Cpanel available, upgrding to this moves us to Apache 2.0 (or 2.2) from 1.8 this in theory wouldnimprove the operation of the server. However because this upgrade has only been out for 2 days I am going to sit and wait and see if others have any issues. SatelliteGuys requires mod_rewrite and ioncube, and I want to make sure those are working ok with the Cpanel update before trying this.
2) Another option is to have some Cpanel exprts come in and run a tuneup on the server. I am looking at two companies who have been mentioned a lot in the cpanel forums. (People have been very happy with their work)
The problem here is its expensive, there are no guarentees they can improve things, and that if we do it, they will work on it when they feel like doing it.
Ultimately I have no problem spending the money even if its not fixed (although I am sure they can fix it based on their reviews from others who had these issues (DBSTalks slow server and the reason they moved to a new server was because their ISP upgraded them to Cpanel 11), the issue I have is I need to be able to setup a time for them to be able to work so they can take the site offline. I just dont want them coming in one day having time and shutting us down with no warning in our primetime hours.
As far as speeds downloading goes, thats nothing I can control. Our ISP is connected to a bunch of the major backbones. If the server is showing page generated in under one second, then the server is working well. I can't control the internet after it leaves our server.
I monitor our server from a total of 4 different ISP's, in addition I using two monitoring services which alerts me of slow page generationand other issues. Whenever any one of these services has an issue I am sent a text message to my cell phone.
I should also note that I have seen issues pop up from time to time when certain maintance kicks in when it's not supposed to (such as statistics processing or sometimes even the backup kicks in) when this happens the system does get slow, but when I notice or am alerted I quickly go in and kill those processes.
So that's whats up.
95% of the time there are no issues, the server functions like it should. Speeds are overall very good to excellent.
5% of the time though there is a slowdown of the system which I have not put my finger on yet.
We run some software on our server called Cpanel, which basically controls the server and helps manage the server. We were using Cpanel 10 and our server loads were below 1.0 (normally around 0.70) then Cpanel 11 came out and something happened there which caused the loads on the server to shoot up and stay around 7.50 or higher. (A lot of folks were complaignin the server was slow... and it was slow because of this reason)
I did a bunch of tweeking when this happend and got the server load down to between 2.1 and 4.1. The speeds picked back up again to an acceptable level.
From where I sit, our server is a powerhouse, we only host SatelliteGuys and DishRetailer on in (Dish retailer is a small forum that normally has 20 - 30 people online.) because of this and the specs with our server I know that something is still not 100% on it. A server like this sould be able to handle 20 or more busy sites like ours without issues.
But it don't, and thats a problem.
There are two options I can do...
1) There is a newer version of Cpanel available, upgrding to this moves us to Apache 2.0 (or 2.2) from 1.8 this in theory wouldnimprove the operation of the server. However because this upgrade has only been out for 2 days I am going to sit and wait and see if others have any issues. SatelliteGuys requires mod_rewrite and ioncube, and I want to make sure those are working ok with the Cpanel update before trying this.
2) Another option is to have some Cpanel exprts come in and run a tuneup on the server. I am looking at two companies who have been mentioned a lot in the cpanel forums. (People have been very happy with their work)
The problem here is its expensive, there are no guarentees they can improve things, and that if we do it, they will work on it when they feel like doing it.
Ultimately I have no problem spending the money even if its not fixed (although I am sure they can fix it based on their reviews from others who had these issues (DBSTalks slow server and the reason they moved to a new server was because their ISP upgraded them to Cpanel 11), the issue I have is I need to be able to setup a time for them to be able to work so they can take the site offline. I just dont want them coming in one day having time and shutting us down with no warning in our primetime hours.
As far as speeds downloading goes, thats nothing I can control. Our ISP is connected to a bunch of the major backbones. If the server is showing page generated in under one second, then the server is working well. I can't control the internet after it leaves our server.
I monitor our server from a total of 4 different ISP's, in addition I using two monitoring services which alerts me of slow page generationand other issues. Whenever any one of these services has an issue I am sent a text message to my cell phone.
I should also note that I have seen issues pop up from time to time when certain maintance kicks in when it's not supposed to (such as statistics processing or sometimes even the backup kicks in) when this happens the system does get slow, but when I notice or am alerted I quickly go in and kill those processes.
So that's whats up.
I'm in the same boat Walt, and it is frustrating. Every other message board I frequent runs fine...just not this one, though it appears to only be a small percentage of us having problems here. From an IT perspective, I can understand Scott's frustration though.
One thing that may help with the Sporadic slow downs, is disable aw stats in cpanel, Cpanel by default has 3 separate stats programs, and when they run once per day usually by default to gather your stats, if you have ALOT of traffic like this site does, those apache log files are HUGE so each day during stats run you will notice your loads go through the roof, sometimes if it was a busy day even enough to crash the apache server causing a restart. I highly recommend against running cpanel on a site with such a huge amount of traffic, for one it causes added load to the server that is really not needed, and for 2 it does alot to limit your ability to control the server itself in the most efficient way. Cpanel is great if your running a web hosting company or a ton of small sites, but a single large site would handle best on a non-control panel server with just apache,
Another thing to note is it may help to setup a cluster, have a dedicated mysql server, on a local network to your server, because mysql is a HUGE memory hog and cpu hog, so if you were to have 2 separate servers one to handle mysql the other apache it would greatly reduce your load and latency on the site as well. Moving to something like this would not be as hard as you think and could take less then a couple of hours for everything to be done.
--edit: Just read the bottom were you mention the stats programs.. instead of having to log in manually to kill the process's you could setup a simple process moniter "pm" script on the server to kill them if they do get out of hand. Id recommend turning all of them off except maybe analog stats. awstats are just way to inaccurate to even bother with, analog is at least semi accurate and least likely to crippled your server.
Another thing to note is it may help to setup a cluster, have a dedicated mysql server, on a local network to your server, because mysql is a HUGE memory hog and cpu hog, so if you were to have 2 separate servers one to handle mysql the other apache it would greatly reduce your load and latency on the site as well. Moving to something like this would not be as hard as you think and could take less then a couple of hours for everything to be done.
--edit: Just read the bottom were you mention the stats programs.. instead of having to log in manually to kill the process's you could setup a simple process moniter "pm" script on the server to kill them if they do get out of hand. Id recommend turning all of them off except maybe analog stats. awstats are just way to inaccurate to even bother with, analog is at least semi accurate and least likely to crippled your server.
It's weird that it's just some of us, some of the time. I'm sure Scott is trying to do all he can and quite willing to do more if anyone knew what that would be.
For myself, I just plain don't know enough about computers (especially internet & network issues) to know if it's something I can fix (or even improve) at my end.
If I knew for sure it was my ISP and could substantiate it, I'd dump it in their lap but they always say if it's just one or two sites, then the problem must be them. It's hard to argue with that, especially if you're as naive as I am about this stuff.
I used to log on many times during the day while I'm working just to do a quick scan of what's going on. I haven't as much lately because I just don't have time to deal with it. Yet there are times I can come on here and zip right along - just not too often lately.
For myself, I just plain don't know enough about computers (especially internet & network issues) to know if it's something I can fix (or even improve) at my end.
If I knew for sure it was my ISP and could substantiate it, I'd dump it in their lap but they always say if it's just one or two sites, then the problem must be them. It's hard to argue with that, especially if you're as naive as I am about this stuff.
I used to log on many times during the day while I'm working just to do a quick scan of what's going on. I haven't as much lately because I just don't have time to deal with it. Yet there are times I can come on here and zip right along - just not too often lately.
The issue is not mySQL nor is it Disk IO the issue is APACHE itself.
Taking a look at the top screen I see this..
Look at all the Apache use, it is eating up the CPU. While mySQL is only using 9% of one CPU (we have 4 CPU's in the server)
I am screwing around with the apache configuration now seeing if I can get a handle on it better.
Taking a look at the top screen I see this..
Code:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
35068 apache 1 129 0 63596K 29040K RUN 2 1:56 72.97% httpd
35006 apache 1 128 0 65536K 30908K RUN 0 1:58 72.19% httpd
35088 apache 1 129 0 65300K 30560K CPU3 2 1:26 71.56% httpd
34990 apache 1 128 0 64268K 29684K RUN 0 0:16 47.37% httpd
35029 apache 1 4 0 58784K 24368K sbwait 0 0:03 4.35% httpd
73308 mysql 271 96 0 692M 483M ucond 2 303:25 9.03% mysqldLook at all the Apache use, it is eating up the CPU. While mySQL is only using 9% of one CPU (we have 4 CPU's in the server)
I am screwing around with the apache configuration now seeing if I can get a handle on it better.
Oh and btw Scott, FWIW I'm changing pages real quick right now. Took the usual long time to get into SatGuys but at the moment (Tues 11:55am) I'm moving great.
Spoke too soon I guess. Now it doesn't want to send this page.
Spoke too soon I guess. Now it doesn't want to send this page.
The delays were probably me, I was tweeking some apache settings and restarted the web server portion to take advantatge of the changes I made.
Those changes may have helped, we were at 8.63 before making the tweek and are now down to 3.26.
I am keeping an eye on it.
Those changes may have helped, we were at 8.63 before making the tweek and are now down to 3.26.
I am keeping an eye on it.
I noticed the site go down for a few seconds too, but it is faster than ever now.
Scott - good job at fixing the problem
Scott - good job at fixing the problem
Yes it does seem much faster. The tweaks seem to be working. Everyone complains about Microsoft updates, but other company updates seem to have as many mysterious problems as MS.
wow Loads are at 3.63 thats insane! Scott it may be time to think about getting another server, 1 server dedicated for mysql and one as just a powerhouse httpd server. The database server doesn't need a internet connection just local to the web server, and the TOP you are displaying shows httpd running because its processing the information from mysql as well. Their are only a few httpd childs going high which tends to say their is a certain part of this site that may be using more resources to access then others. Also if you notice mysql is using about 700MB of ram, Adding more ram may help as well, if you are using a large portion of Swap memory... Ram is alot faster then a HDD...
but yea, 300% cpu use is a bit much
MySQL = Memory hog
PHP = spu hog
VB = Heavy php and Mysql = OMG!
but yea, 300% cpu use is a bit much
MySQL = Memory hog
PHP = spu hog
VB = Heavy php and Mysql = OMG!
- Status
