Shared hosting: Pay Peanuts, Get Monkeys

See the bottom of the post for updates on my adventures with shared hosting..

My experience in selfhosting my blogs with GoDaddy moved from a glowing enthusiasm via consternation and frustration into a deep distrust and disbelief. In this post, I want to take you through the past year, as I discovered some of the issues one should know before choosing a selfhosting service.

Much is written about hosting your own blog and how to choose your hosting company. Whole blogs are dedicated to attracting customers to this or that hosting supplier, a clear sign hosting is big business.
I will not analyse which hosting company is the best, nor what the pros and cons of all hosting companies or hosting formulae are. Others are more qualified to do so. But… as with most items we cover on BlogTips, I want to share my experience, hoping others will learn from my many mistakes. My experience concentrated mainly around GoDaddy.com, one of the biggest and cheapest hosting companies around.

My first steps into the hosting world:

I have been blogging for over three years. My first blogs were hosted by Blogger, WordPress, Skynet and Tumblr. Later on, I also started some blogs on Posterous. About two years into blogging, I thought it was about time to take more control over my blogs and to selfhost some of them.

As with many things in life, I found myself walking a road, unconscious of the direction I took, and soon enough, found myself in a middle of a mediaeval battlefield with all kinds of things happening I had no control over, no influence on the outcome, nor did I have the faintest clue “how the **&&%% I got there”. Let me explain: I choose GoDaddy.com as my hosting provider. It was not a calculated choice, but Godaddy was the service Google used as a registrar when I registered the domain of my very first blog. The migration from a Blogger domain to my own, went fast and transparently, so I thought GoDaddy would also be the way to go for hosting services too. Little did I know that domain registration and hosting services are two completely different things.

Via GoDaddy’s home page, I found a link to their selfhosting services, and registered. Not really thinking what I choose for. “Linux economy plan” smelled about right. And at less than US$4/month for 10 Gb of diskspace and 300 Gb/month of traffic, what could go wrong? Right? So before I knew it, I started walking the road of “Shared Linux Servers” selfhosting.

PLUS: shared hosting services are cheap
PLUS: shared hosting services provide loads of diskspace and high internet traffic quota

Starting my first selfhosted blog Have Impact was a breeze. When I registered the domain, I could select the hosting service I wanted. The DNS entries – the link between the domain and the physical space where my blog lives – were done automatically. Installing WordPress was just a matter of clicking a few buttons and I had a blog up and running in minutes.

PLUS: a fast and transparent link between the domain registration
PLUS: an easy “Pay and Go” solution which brings up a website in no time
PLUS: easy and fast installation of the blogging software

I thought: “Why did I not do this earlier?”

A clear blue sky

Using a selfhosted WordPress.org blog is pretty much like using a blog on WordPress.com, where your blog is hosted by WordPress themselves. But it gave me more freedom to install and tweak themes and plugins, making my blog do exactly what I wanted.

All fine so far. Apart from a bit of a surprise how much time it took to ensure my blogsoftware, themes and plugins were kept up to date with the latest software releases, I was a happy camper: My blog was puttering along nicely, with a few dozen posts, updated once or twice a week, and a few thousand visitors a month.

PLUS: For a low traffic, simple blog shared hosting services are just fine.

So I continue walking on that path: I created several other blogs hosted by Godaddy, amongst which your very own BlogTips, this blog. Before I knew it, I had a dozen selfhosted blogs, all on GoDaddy. I even started a Drupal site, Humanitarian News.

The first signs of trouble

The latter took off into cyberspace like a rocket, with loads of content and more visitors every month. It was also the first site I experienced problems with. Problems I would also encounter with my other blogs later on: The increased traffic and the amount of posts I published forced me to tweak the site continuously: I applied aggressive caching and stripped it off all functions that had a bad influence on its performance. Soon I was forced into the Drupal internals, PHP/MySQL tweaks and all other stuff a naive blogger like myself should keep his hands off. Really… Soon enough, I needed help. As GoDaddy promised 24h/7d telephone and email support, I thought nothing could go wrong.

I learned their Email support did not work well. Most of the time, the answers I got were off-topic, or not to the point, and looked like pre-cooked “cut-and-paste” replies. It honestly felt like they let you go through a couple of to-and-fro iterations to ensure you are persistent enough, and you “really have a problem”, before you do get a real answer…

As most of my issues were performance related, most of the time, their answer was a standard “we see your site might be loading a bit slow, but we advise you to”… and then came an explanation on how to increase the speed of a website by compressing pictures, etc… Even though none of my performance issues had anything to do with the web-side (or front-end) of the chain of events, but were rather concentrated on the back-end.

The more I insisted, the more frequent the “end the discussion now”-killer sentence came: “We don’t see any performance issues on your shared server, but if you want better performance, we suggest you upgrade to a hosting package XYZ”… Which implied moving all my data myself, at a significant higher price tag. Beh..

Shared hosting = shared trouble

As time went by, I discovered I took the cheapest and most unreliable of all packages: a shared host. Meaning, I shared the machine my blog ran on, with thousands of other users, rather than having a machine to myself. Other formulae were “dedicated virtual hosting” services (where you run on your own virtual machine) or “dedicated hosting” (where you run on your own physical machine)…

Minus: Shared hosting is not a good idea for high traffic sites
Minus: There is little or no performance to be expected from shared hosts
Minus: Email support seldom leads to actual solutions. Real support is only possible via telephone.
Minus: Most Email support mostly consists of standard cut-and-paste answers.

The more I used the support services, the more I discovered problems: each time I called or emailed, I got another person on the line. Who might or might not do the effort of reading through the history of problems. Who might or might not be knowledgeable. Who might or might not be interested or motivated to really help you.

Minus: There is not a single focal point for each account, nor a focal team. The quality of support really depends on the person.

Consumer rights

Around late last year – I also discovered other problems. The top of the iceberg, it seemed. One of my websites ran significantly slower than all others. The lack of speed had nothing to do with my blog itself, nor the posts: Just about anything I did in the WordPress dashboard was at least ten times slower than on any of my other blogs. I found out the IP address from the server on which “the slow blog” was running, was in a different range from the servers of my other blogs. I gathered it ran on a different server array, or even ran in another location. That seemed like an obvious explanation of the source of the problem: the slower website must have a server with a resource problem: CPU, bandwidth, memory, load etc…

It was by then I also started see some variations in performance on my other blogs, including this very same blog, BlogTips. At times the site would load THAT slow, browsers would time out. WordPress admin functions would take minutes rather than seconds. The problem would persist for about half an hour and by the time I called or emailed support, the problem disappeared… To re-appear a few hours later, or on the following day, or the following week, or the following month.

Next thing I knew, the CRON jobs on Humanitarian News stopped working. CRON is the mechanism scheduling and executing background jobs such as indexing your site for the internal search. CRON is particularly important on Humanitarian News as it also imports posts from RSS feeds. No CRON, no new posts…
Once more the answer from GoDaddy’s support desk was similar to previous issues: at first there was a denial there was a problem, but when I persisted, they confirmed there was indeed a problem, but were unable to tell me when the problem would be resolved. Nor could they inform me by email when the problem was resolved. This also made me realize there is no system to actually ‘close’ a support ticket. A support issue seems to be closed the moment the client gives up, or when the answer is ‘we know of the problem, and it will be resolved. Bye!’…

Minus: When there is a recognized problem that can not be cured on the spot, support often can not give you a time frame when it will be resolved, nor will they contact you when the problem is actually resolved.
Minus: There is no system to close support tickets

That made me think about consumer rights… I guess I had only one right: To leave… But for the rest, it seemed I had no way to escalate a problem, or to appeal to what I felt was unfair consumer treatment. Imagine you buy a car, and the engine stops. You can drive it downhill, but for the rest, you won’t get far unless if you push it. And the garage won’t be able to tell you when the problem will be resolved. Would you accept that?

The first step to recovery, is to recognize one is ill

Despite repeated emails to the support service, little I could do to convince them about most of the performance problems. True, I used to manage DEC VAX and PDP systems twenty years ago, but Linux is not my thing and I have no system tools at my finger tips. I could not measure the actual performance of my site. But to my surprise, it seemed they did not have much of the tools neither to measure, monitor or benchmark performance issues on shared hosts. And even less tools to analyse and/or cure the problems. I witnessed  little eagerness, willingness (or ability?) to check performance logs.
Nor could anyone tell me what performance I could expect. What metrics were used? I figured I basically signed a blank cheque.

Minus: There is no benchmark or agreed performance to be expected from shared hosts.
Minus: There is hardly any action taken on performance complaints

On some single occasions, I had GoDaddy admit there were actual problems on my shared server, but then their answer often was: “In the mean time, the problem has been resolved, please revert if you experience similar problems”.

It was then I also discovered:

Minus: there is no refund for downtime, nor a money-back guarantee for dis-satisfied customers

Signing a blank cheque

As Humanitarian news continued to grow, I learned there were several system limits which are undocumented or rather grey-ishly documented: the fact that you can have 10 SQL databases per hosting service is fine, but none of them can be larger than 1 Gigabyte… If it grows larger than 1 Gigabyte, you can not back it up. Punto. It was also difficult to find what the maximum memory size. And there was no way to increase any of these. Virtual shared hosting formulae were not modular in design. Take or leave the standard formulae…
It was also difficult for me to analyse the problem as I had limited access to error logs.

Minus: Undocumented or poorly documented limitations of the hosting service.
Minus: No PHP-error log access.

Hackers aboard

Then came the April-May debacle of the massive shared host hacks. Several of my GoDaddy hosted websites got infected. Even though I cured the sites quickly, several sites went down repeatedly. I spent hours curing the infections the hackers left behind. Meanwhile Godaddy did not admit guilt, and continued to point the finger-of-blame towards WordPress and the individual users.
I discovered how little repercussion the clients of (shared) hosting services really had. None of the hacks happened on dedicated hosts, the security holes apparently only happened on shared hosts.

Minus: Shared hosting services are more vulnerable to hackers’ attacks

I discovered how little support was given for the applications running on the hosting service: few of the support people actually understood Drupal, or WordPress, or Apache, or MySQL…, something which is not clearly stated when you sign up for a hosting service:

Minus: Application support is limited.

The past few weeks, the problems with Humanitarian News got from bad to worse.The SQL server completely failed on several occasions, it timed out most others. This caused all kinds of database errors, which took days and days to resolve. I got mixed up into a whirl of snowball effects…

The apotheosis

It felt like the support desk would no longer put up with my persistent complaining on the lack of performance on their services: A few days ago, I received an email my website was put offline “as I used up too many resources”:

It has come to our attention that your humanitariannews.org hosting account, specifically the humxxx database is causing the shared resources to be over-utilized. This, in turn, affects the usage by other customers.

We have disabled your database to return the server to normal usage. To re-enable your database, you will need to correct the following query:

Access is revoked. Problematic query:
SELECT t.word AS realword, i.word FROM search_total t LEFT JOIN search_index i ON t.word = i.word WHERE i.word IS NULL

EXPLAIN:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t index PRIMARY 152 266794 Using index
1 SIMPLE i ref word_sid_type,word word 152 humxxx.t.word 29 Using where; Using index; Not exists

This query examines 7737026 rows, which is unacceptable in shared hosting.

Please respond to this message via email or phone with the steps you will take or have taken, to correct this issue, and access to your database can be restored.

Admittedly the timing of this was rather weird. Anyways, I figured out the offending query was caused by the standard Drupal “Search” function. I disabled it, and it took GoDaddy 20 hours to bring the website back up. I asked them what the metrics they used to determine what resource consumption was acceptable and what was not. I am waiting for the answer.

Minus: There are unclear metrics used on the use of shared resources

I was given administrator rights to the entire server.

Interestingly enough, when they gave me back the access to my own database, something really weird happened: I went into PHPMyadmin, to look at my SQL database and did not find the usual two databases, but over 2,400 databases. I clearly had access to the databases from all other users on my shared server.

And I found dozens of processes on my server too:

By error, I was given access to all 2,400 databases and all processes on my shared hosts. I could delete 2,400 websites right there on the spot. Or I could play cat and mouse, and delete random SQL processes, and see how long it would take before Godaddy caught me.

Honestly lasts longer, though. I called them. Was put on hold for 12 minutes. Talked to a support guy. Who did not really seem to believe me at first. Then forwarded the request to the hosting experts, and thanked me for reporting the problem.

It took them a few hours to take away my “administrator” access on the shared server. This did not particularly strengthen my belief in the rigidness of the shared server management. And, just as before this incident, I could still read everyone’s access log (the log registering who does what on a website). Godaddy is still checking if ‘that is normal’ or not. (Update: GoDaddy support confirmed it was normal I could read everyone’s access log)

Access logs for a site called "humongous"

Minus: “Shared hosting” and “security” are two words that should not be used within the same sentence.

I *did* get an email from “the office of the President” – that is the president of “GoDaddy.com”, and not of “the United States”, to thank me for reporting the security problem, saying they tried to call me to see if the issue was resolved.

I wrote them back stating the only way they could show their gratitude to me, and to their thousands of customers, was to monitor and cure their performance problems on the shared hosting servers…

So what did I learn from all of this:

• There are many hosting formulae, and many hosting suppliers. Choosing the right one is as critical as choosing the plot of land on which you will build your house.
• GoDaddy Shared Hosting Services are only good for low traffic, low demand websites.
• Technical support from Godaddy is limited, and only effective when you call.
• If your website is critical, and you want a guaranteed performance and uptime, shared hosting is not what you want. Certainly not from Godaddy. Go for dedicated (virtual) hosts.

The bottom line:

Am I pissed off? Kinda.. Mostly on myself. What kind of support would I have expected when paying US$4/month? Indeed: when you pay peanuts, you can only get monkeys.

I am now looking for another hosting formula, on another hosting provider. I will keep you in the loop of my discoveries in “Blogging Never-Ever-Land”.

Updates:

1. One day after publishing this post, once again, BlogTips.org gave time-out errors. After half an hour online with the support services, they admitted there was a load problem on the server. They said it would typically take 3 days to resolve.
   Meanwhile another site gave “500 Internal Error” problems, which turned out to be a DNS setting which was changed. How? I still don’t know. — GoDaddy confirmed later on that this was a new option people could set. Apparently there was a problem in the migration.
2. Meanwhile, it seems that the option causing the DNS error is taken away from all hosting accounts…
3. I continued to experience problems, mostly time-outs when loading pages, but also performance problems with the SQL database. Godaddy support suggested I moved my shared hosting accounts to grid hosting, a process which can be done by the click of a button.
   Indeed, the migration process was fast and flawless.
   Let’s see if the performances gets better.
4. After migrating all but one of my hosting accounts to grid hosting, most of them started to give time-outs upon loading. Godaddy let me do a TRACERT to my domains, which showed on all but one, time-outs.
5. After 5 days of “we’re working on it”, I got a mail saying “Solved!”, but it failed at the first test. After that, for three days, I got “Working on it, but can not say when it would be resolved”.
6. The rest of my sites started to give “500 Internal errors”. I can’t wait for this holiday to end so I can migrate all my hosting accounts off Godaddy.
7. On July 20, Godaddy publishes an update on their support website: “We’re aware of an issue within our Grid Hosting services impacting a few customers. While we’re fixing it, you might experience longer than normal phone support hold times. Thanks for your patience.”
   ..a few customers… Right…
8. GoDaddy confirmed by telephone that TRACERT of one’s own domain should give a time-out as soon as the route enters their domains, as a security precaution. Makes me wonder then why I can do a TRACERT of all my domains I tried, except the one for the site which was unreachable… Beh.
9. After much to-and-fro, including an email exchange with GoDaddy’s “Office of the President”, GoDaddy moved BlogTips to another server. Since then, BlogTips has been more stable. This morning I wanted to log in to post this update, but the site would not load again. Sigh.

Pictures of “Tintin and The Seven Crystal Balls” and “Tintin, Destination Moon”, courtesy Editions Casterman.

Tagged as: hackers, hosting, tips