Only by reading the TOS will you discover, for instance, that many shared hosts have policies against installing things like chat software, or that some companies won’t allow you to offer downloads for your readers (although there are ways around each of these problems as we’ll see in later posts).  But what is really important to learn is the factual information about the server resources you will be allowed to use.  Remember that on shared hosting, your site will share a server with many other sites.  So your site will be expected to behave in a certain way in order not to bog down all the other sites.  For instance, having “unlimited bandwidth” doesn’t mean anything at all if you are on a server with 200 other sites and will be shut down if you use more than 5% of the processing power.

Many times, however, even the Terms of Service are written in vague terms, as the hosting company doesn’t want to be pinned down.  For the most part, when all you find in the TOS are more vague promises, then it is time to move on to another company.

Let’s look at a real world example, though with the names of the companies removed and the legalize has been paraphrased.  Which hosting provider would you choose?

Company A:  We place no limit on the bandwidth which may be transferred from an account each month, but we do have to protect all customers who share a server.  We sometimes work with customers who have special needs to place them on larger boxes or boxes with fewer sites.

Company B:  No hosting account may use more than 25% of server processing resources for longer than 15 seconds at a time.  Customers who consistently use more than their allotted resources will be contacted to upgrade their account.

As I said, the above excerpts of were paraphrased from actual companies, two of the largest hosting companies in the world.  Which would you choose?  You have to pick Company B.

While Company A is well-known and seems to be going the extra mile by saying that they sometimes work with heavy-use sites, the reality is that this is a far cry from a real, in-writing solution.  You are not given any information at all about their criteria for excessive use.  In a real-world test of Company A, I discovered that a dynamic site that uses the database frequently started to fall apart after only about 100 hits in an hour.  When this sort of use became consistent, an by then the site was terribly slow, it was simply taken down by the hosting company with no offers to move the site to a “larger box.”

Meanwhile, Company B almost seems harsh in their TOS compared with Company A, but here you have, in writing, precisely what you can and cannot do.  You can even track how you are doing within your control panel.  So when that day comes, if it comes, when you begin to hit the traffic jackpot, you’ll be able to see that you are getting close to maxing out your welcome on the current plan, and you’ll know what is coming next;  you’ll be asked to upgrade your account.

So when checking around for a hosting company, take a side trip away from their marketing material and visit the TOS.  If it is just as vague as the marketing materials, take your business elsewhere.

While this is good information for everyone, for this post, we’re going to look at bad bots from a WordPress point-of-view.  In other words, you have a WordPress installation on your own hosting account.  Everyone with a hosting account, however, should know this information and future posts will show you how to stop these sorts of attacks on any type of site.

With every host, you pay for cpu resources and bandwidth.  If you are with a shared hosting company, even if they offer “unlimited” bandwidth, you only get so many resources.  Did you know that there are people out there trying to steal those resources from you?

One thing many people aren’t aware of is that there is a huge and shady industry out there designed to search the web with automated programs that try to accomplish various things.  These bots go around and do stuff, or attempt to do stuff, to your site.  One common bot is the email bot.  An email bot will search the web looking for email addresses either in posts or inside comments.  Once it finds an email address, it shoots it back to its server where it becomes the property of spammers.  If you’ve ever posted your email address at a website and suddenly started getting spam, it wasn’t really that site’s fault.  It just happened to come under an email bot attack.

Another type looks for forms that post to the site and try to post spam that way.

Other bots try to leave spam in your comment sections.  You may think your site is okay because your anti-spam software filters out the spam comments.  Chances are that you are wrong.  Just filtering out the spam isn’t good enough.  Here’s a real life example of why:

On one of my sites that is hosted by a large, grid hosting site, I keep track of all of my CPU usage to see if I may incur any overage for the month.  Too much overage, and I’ll start splitting sites out into different accounts.  So I was checking my CPU usage the other day when I came across something shocking.  One of my lowest volume sites, a site that gets no more than about 50 real visitors a day, was logging, by the hosting company’s stats, several thousand hits a day and burning through my server allotment at an astonishing rate.  I had noticed that a couple of spam comments were getting through my Akismet spam protector every day or so, so I went to my site and checked my spams blocked stats.  I was shocked.  Akismet was blocking thousands of spam comments every day. 

That’s not so great.  Why?  Well, because these bots were still accessing my site.  They weren’t being successful at posting, but they were still pulling my site up and wandering around the pages.  They were using forms to post comments.  They were burning up my server.  If this site had been hosted on a shared host, it would have been taken down.  That’s how severe it was.

For any site, you can block bots by adding some code to your .htaccess file.  But that will take some explaining and will come in a later post.  When I get that done, I’ll put a link to it at the top of this page.  For WordPress sites, on the other hand, there is a good solution in the form of a plugin called Bad Behavior.  Simply install it and Bad Behavior will stop most of the bots before they are even able to access the site.  Instantly, the burning through my server usage came to an end.

There is one slight warning to go with Bad Behavior.  You need to check your main page in Google, and maybe Yahoo, at least once per week.  There have been times, though very rarely, where a Google bot will get associated with bots from a bad neighborhood and then Google will be unable to access your site.  That’s very, very bad, but also very, very rare, and the developer of Bad Behavior is quick to track and fix all problems.

So what is the bottom line here?  Don’t let the bad guys steel your server resources.  If you are on a shared host, especially, the consequences could be catastrophic.  You could be running a site that easily meets the guidelines of the shared hosting company, and you could be shut down anyway, and if you have never heard of these nefarious bots before, you would have no idea why you are being treated this way by your host.

So always make it a habit to check your logs.  Bots don’t show up in most stat programs, like Google Analytics, so you can’t see them this way.  If you have Akismet installed on WordPress, check to see how hard it is having to work.  Otherwise, log into your host’s control panel and look carefully through the logs.  Bots are pretty easy to identify.  They are indiscriminate in the way the navigate your site, going to everything every time they show up.

This is some of the most dispicable behavior on the Internet and can literally cost you your site.  Don’t let it happen.

Index

The Six Reasons Your Site is Slow
   You’ve picked a lousy shared hosting company. 
   You’ve got a crazy site. 
   Your site has been targeted by large hosts of spam bots.
   Your traffic is too much for a shared hosting account.
   You have errors or your databases and tables need to be repaired and optimized.
   Any combination, or all, of problems 2 through 5.
How to Go About Identifying and Fixing the Problem
   Traffic
   Checking for errors and repairing and optimizing your database
   Checking for and getting rid of spam bots
   Optimizing Your Site
      Images
      Caching
      Eliminating Processes
Final Evaluation

Is your shared hosting company driving you nuts?  Is your site consistently slow, freezing up, and doing all sorts of nasty things that you would much rather it not be doing?  After we go through each of the potential problems, I’ll tell you how you can figure out where your problem is and whether you have a lousy hosting company or not.  It is actually fairly easy to do.  The six most likely problems are:

1. You’ve picked a lousy shared hosting company. 
2. You’ve got a crazy site. 
3. Your site has been targeted by large hosts of spam bots.
4. Your traffic is too much for a shared hosting account.
5. Your databases and tables need to be repaired and optimized.
6. Any combination, or all, of problems 1 through 5.

1.  You’ve picked a lousy shared hosting company. (Return to Index)

Hey, I’ve done that before, so it isn’t your fault.  You bought the marketing and opened your wallet.  Some of the biggest names in the business are lousy, sticking your site on a server so jam packed with sites that it can barely function even when traffic is low.  Or, your are somewhere where they specifically throttle your service, meaning that you can never have great performance with their shared hosting.  You can’t make the hosting company better, and I wouldn’t reward them for lousy shared hosting by upgrading to VPS or dedicated packages, so your only option here is to move to a different company.  But before you move, you need to make sure that your hosting company is at fault.  If there is something inherently wrong with your site, moving it to another host is just taking your problems with you.  If you are at a host that has a reputation for solid performance, like GoDaddy, HostGator, or Pronet Hosting, moving to another shared hosting company probably won’t fix your problem.  In any event, a move can sometimes be painful, so even if you are at the worst host (won’t mention names in this article), it’s best to try to fix all your problems before undertaking a move.

2.  You’ve got a crazy site. (Return to Index)

This is common, particularly if you are running your site with something like WordPress or Drupal and using a lot of plugins or a theme that was not professionally developed.  Some plugins for WordPress, for instance, cause the server to run back to the database as many as thirty or more times just to pull up a single page.  And then there are the free themes, or themes that you’ve extensively modified, that are written with poor coding, where cycling through the database multiple times is just a fact of life even if you have no add-ons.

Of course, whether you have a static or dynamic site, coding errors, countless javascript functions, complex flash files, and large photos can be enough to do you in.

But if your site is going slowly, the first thing to do is to look for errors–scripts, for example, that are looking over and over for things that aren’t there.  Almost every hosting company has a control panel you can use to track errors. 

The next thing to do is to simplify your site.  A great site is one that is elegant, simple and easy to use, not one that tries to shove a free cappuccino through the user’s screen (although I would visit your site every morning if you could pull that one off). Don’t put so many pictures on the main page.  Get rid of some “bells and whistles.”  Look for plugins you can deactivate.  With most hosting companies, you can go into your control panel to see what processes are running and get a pretty good idea of what is going on.  One thing, going back to the WordPress plugins, that causes a lot of problems are scripts that look for posts by a certain ranking; for instance, most popular, possibly related posts, etc.  If I’m having problems with a WordPress site and don’t have any errors, those are the plugins that go first.

If you have a static site with a lot of pictures, make sure that they are optimized.  You would be surpised at, using jpg files, how much space you can save just by dropping the quality 10 to 25 percent, and the vast majority of the time you can’t even tell a difference in the pictures.

You can also try moving videos and flash animations off of the front page if they aren’t absolutely necessary there.

3. Your site has been targeted by large hosts of spam bots. (Return to Index)

This one gets a whole lot of people and they never even know it.  In fact, I’m getting ready to do a whole series on spam bots and the problems they can cause site owners.  As an example, I recently discovered that one of my least popular sites was one of my busiest.  How is that possible?  That site got targeted by spam bots, who were accessing every article on the site and trying to add spam comments.  The site, which gets about 50 real visitors a day, was logging several thousand hits by bots.  After blocking the bots, my server usage went way down.

This is something you really need to keep an eye on.  Most statistic programs, like Google Analytics, are trained to ignore bots.  So you can look at your stats and not see that you are getting worn down by them.  What you need to do is either install a tracking program for bots or keep an eye on your log pages.  Or, you can just skip that and get rid of the bots.  This is a delicate act because the only way you get into search engines is to let their bots scan your site, so you have to really know what you are doing in order to block one type of bot and not the other.  But that’s why I’m going to write an entire series on the subject, though I will go a little further into this in a section below.  Just know that you need to keep an eye out for being overrun by nefarious bots.  You can easily be kicked off a shared host just from the actions of spammers.

4. Your traffic is too much for a shared hosting account. (Return to Index)

Depending on your site construction (see number 2 above), you can quickly run out of server resources at a shared host as your traffic increases.  But even if you have a single, static page, it is conceivable to eventually get too much traffic for a shared host to handle.  If you are getting lots of traffic, and you’ve analysed potential problems 2 and 3 from above, and you are still going too slowly, it may simply be time to move on.  Generally speaking, at a good shared hosting company, you should be able to get a couple thousand hits a day on a well-built dynamic site, and several thousand more on a well-built static site.  If you are new to the world of web hosting, by the way, any more than 1000 unique visitors per day is a very good number.  The overwhelming majority of sites don’t get anywhere near that.

5. You have errors or your databases and tables need to be repaired and optimized. (Return to Index)

If you run a dynamic site and add and delete content from time to time, you probably need to repair and optimize your databases about once per month.  Remember to back up everything before you do this.  If you don’t know how, don’t worry; we’ll go over that in just a minute.  I will say that databases have to be pretty seriously compromised, from my experience, before they cause real problems.  Unfortunately, this is unlikely to be your only problem.

6. Any combination, or all, of problems 1 through 5. (Return to Index)

You need to realize that all of the five items above can be working together to slow your site down. That’s why you don’t jump to conclusions and, instead, methodically determine what it is that is causing the problem.  Which brings us to the next part, which is how to determine what your problem is and how to fix it.

How to Go About Identifying and Fixing the Problem

(Return to Index)
Traffic

Before you go nuts with the next sections, although everything below should be done no matter if you are having problems are not, you should realize that, realistically, if you are getting a few thousand unique visitors a day or more, you have probably outgrown your shared hosting account unless your site is very simple and doesn’t require much of the server.  If you aren’t getting that much traffic, and you are having a problem, try all of the below until the problem is solved.  If you do everything below, and your site is still slow and you don’t get that much traffic, then you are with a bad hosting company.

On the other hand, if you are getting very good traffic and things are starting to slow, you may be able to get a lot more miles from your current host by following the advice below.

Checking for errors and repairing and optimizing your database (Return to Index)

So, given the information above, what’s the right way of going about figuring out where your problem is?  Well, I assume that you like your site just the way it is, so let’s start with the non-invasive testing.  Let’s look for errors and spam bots.

There are as many control panels as there are hosting companies, but the most popular control panel, particularly with shared hosts, is cPanel, so let’s look at that.  Log in to your cPanel and towards the middle of the page you will see a section called “Logs.”  First of all, click on “Error Logs.”  You should see nothing at all.  If you see errors, it should give enough of a description for you to figure out what the problem is and go fix it.  Fixing any errors is your first step.

Now you should repair and optimize your databases.  Don’t worry, it isn’t as hard as it sounds.  Back at the control panel, go down to the section titled “Databases” and click on “phpMyAdmin.”  Click on “Databases” and then select your database.  Now you will be at a screen that shows all the database tables.  Select all the tables and then, from the dropdown box at the bottom of the page, select “Repair Table.”  Now all your tables have been repaired.  Now go back and select all the tables and then, from the dropdown box, select “Optimize.”  Easy as that, you are repaired and optimized.

If your site has just been a little slow and you’ve corrected errors and optimized your database, now might be a good time to check and see if your performance has improved.  If it hasn’t, let’s keep going and look for spam bots.  Actually, it’s a good idea to look for spam bots even if your site seems to be going well.

Checking for and getting rid of spam bots (Return to Index)

Back at your control panel, you should find an Awstats option in the “Logs” section (if you don’t, go toward the end of this section for an alternative).  Click on it and then click on the magnifying lens next to your site name.  This will bring up the stats page.  DON’T FREAK OUT.  If this is the first time you’ve seen your stats with every single hit, including bots, you are probably in for a major shock.  Just remember that a lot of bot traffic is genuine.  You want Google and other search engines to be sending their bots by frequently. 

Towards the middle of the page, you’ll see a “Hosts” section.  This is supposed to be real users, not bots, but bots frequently show up here.  If you are an active webmaster, the first few positions, the IP addresses that have visited the most, are probably yours.  Copy the IP address and paste it into a Google search.  You’ll probably see that this IP comes from your home town!  What a coincidence, huh?  Anyway, what you are looking for in this section is IP addresses that aren’t you but which have accessed the site numerous times and just seem, generally, odd knowing what you know about your traffic.  Take any IP addresses you have a question about and search them in Google.  If it’s a bad bot, there will likely be many people complaining about it.  Make a list of all the bots you don’t want visiting your site, if you have any.  Then go back to the cPanel and in the “Security” section, select “IP Deny Manager.”  Enter all your unwanted bots into this.  If you are having bot problems, you’ll need to check this every few days for awhile to get it completely clean, or you can enter a range of IP addresses if it is apparent that you are getting the same bot from a dynamic IP generated address.  One thing to look for, if you are an American site, is for bots from odd places.  Why should someone in Africa be visiting your site over and over again?

If you don’t have Awstats, or something equivalent in a non-cPanel environment, then you may have to take a longer look.  In cPanel, you can look at the “Latest Visitors.”  This is tedious, but you will basically want to look for IP addresses that have visited your site and gone all over the place.  Copy the addresses and paste them into Google to see who they are.  If they are someone you don’t want and don’t seem to be a real person (very rarely will one person show up at your site and go to every single page), then put them into the IP Deny area of cPanel.

If you discover that you’ve had a serious problem with bots, block them and see what happens.  You may see nearly miraculous results.  Just remember, you don’t want to block any bot that might be a search engine bot.  Once you start denying IP addresses, pull your site up on search engines every now and then to make sure you haven’t accidentally blocked a good bot.

Optimizing Your Site (Return to Index)

Images

So you’ve done the above three steps and things aren’t working out for you.  Your site is still slow.  Now it is time to try optimizing your site, as painful as that is.  The first thing you can do is look at how graphic intensive your site is.  Do you have graphics up?  Most sites do.  Take the first picture you come to and right click on it, unless you are a Mac user then I have no idea what you do.  After you right click, you should get a properties box and you can see how large the picture is.  In the old days of dial up Internet access, you really wanted less than 100k, a lot less most of the time, worth of pictures on a single page.  You can do a lot more than that now, but you still want to keep it to a minimum.  If your home page is measured in megabytes instead of kilobytes, you’ve got very serious problems.  There are lots of free sites on the web that will test your site speed and give you all the details.  Just do a search and run a test.  Here’s one good place to go: http://www.websiteoptimization.com/services/analyze/

As I said in a section above, you can easily optimize your photos in just about any graphics program.  If you are using jpeg files, you can drop the quality down on each photo, save a ton of space, upload them back to your website and no one will know the difference except that your site will load faster.  Properly optimized, pictures can take up a lot less space and look essentially the same as they did before optimization.

Caching (Return to Index)

If you run a dynamic site that accesses the database when visitors come by, install a caching plugin.  Through the use of caching, you can sort of turn your page into a virtual static page.  Intead of accessing the database over and over again, now your site will keep pages stored and serve up standard HTML pages to your readers.  Of course, you can set the pages to expire whenever you want them to, to expire every hour or to expire when they have been changed, for instance.  Nearly every major software platform for running a dynamic site, from Zencart to Drupal and WordPress, offers caching plugins.  Get them.  Use them.

Eliminating Processes (Return to Index)

Most sites these days are dynamically generated, relying on information pulled from databases.  How many processes do you have running just to get your page up?  And even if you don’t have a database driven site, chances are that you still have processes running.  Do you have rotating headers?  Do you list the same content in multiple ways on your main page when it isn’t really necessary?  Are you generating things like “Most Popular,” “Recent Comments,” etc?  Do you have multiple stats programs running in the background?  Take a careful look at your page from a process point-of-view, both on the outside and the inside, and see if you really need everything that is there.  If you are using a popular software format like Drupal or WordPress, do you have inefficient plugins that are accessing the server over and over?  Basically, consider that every plugin has a cost and then decide whether that cost is worth it to you.  In fact, every process you have that generates something on the page has a cost.  Remember that you are on a tight budget with a shared host.  If you are running slow or using too much bandwidth, start being pickier with the processes that you run.  For instance, instead of having “related posts” or “most popular” posts generated automatically, consider putting exactly what you want there with HTML.  Manually pick the posts and code them in.  That’s a much more efficient process for the server.

Now look at all your fancy stuff, if you have any.  A lot of new designers go as crazy as possible, with lots of rollover effects, etc.  These all have costs, too. 

If your site is fast and you’re having no bandwidth problems, you can afford all these processes.  If not, you have to start budgeting your server allotment better and start to eliminate things.

Final Evaluation

(Return to Index)
So, you’ve eliminated any errors; you’ve blocked all the bad bots; you’ve cleaned your databases; you’ve optimized and maybe even removed images; you’ve started using web caching; and you’ve reduced the number of processes running on your site.  If your site is still slow, it’s time to move.  The question is whether your site is going to be slow at another shared hosting company or whether it is time to move up to VPS or grid service.  Here’s a rule of thumb to follow, though this isn’t set in stone.  If you’ve done everything above, and your site traffic is around or below 1000 unique visitors a day, you are at a lousy shared host, and you can look for another, better one.  If you’ve done everything above and you are getting around 3000 unique visitors a day and you are still slow, it is time to move up to a more robust hosting plan.  But what about if your unique traffic is between 1000 and 3000 each day?  This is where we get into a very hazy area.  It really just depends upon how demanding your site is. If you have a static site with few pictures and scripts, you are definitely okay to look at another shared hosting company.  But most sites aren’t like that.  Most sites are more complex.  If you’ve really done all that you can do to minimize the amount of effort your server and you are closer to 1000 unique visitors a day than you are to 3000, then you may well want to try another shared hosting company, like HostGator or Pronet Hosting, to see if you can do better, but there is a risk that this move will be a waste of your time.  If you are really getting good traffic, and over 1000 uniques a day is good traffic, and you are very committed to your site, then I would suggest that you start shopping for a more powerful hosting solution, like VPS, dedicated or grid hosting.