How one small plug-in can slow down your site.
The Drupal version
Before I begin, let me re-state that Drupal by itself, is fast. I mean real fast. That’s why I gave Drupal two thumbs up in my 2010 blogger’s wrapup. But it only takes half a dumb system administrator (or web admin) to make any fast system slow. And I succeeded very well in that.
As I was debugging my server’s CPU load for my WordPress blogs, I saw that my Drupal site, hosted on the same machine, was putting a lot of pressure on my SQL and Apache servers.
Well, the site as such was not slow. As I used extensive caching, a visitor did not see anything. But boy, the site really loaded my server!
I did not understand why. I mean, sure, I have a lot of traffic, but as with most news site, 90% of all traffic goes to the same most recent posts. And that was caught by the cache, so that traffic should not even touch PHP or SQL. Proof of the matter was that as a visitor, the site was very fast, taking only one or two seconds to load.
So why did the site put that much load on the server? I found out be pure luck.
I was checking Google Webmasters how the sitemaps for my different blogs were going, and noticed my Drupal site’s sitemap never gets submitted to Google. So I looked at the module settings. Something I should have done much earlier.
Apart from the module having apparent problems with high volume sites, I also misconfigured it: I had set the module to regenerate a sitemap of the latest 2,000 posts at every CRON (the internal scheduler) run.
On my site, CRON is set to run at every 1o minutes, as I use CRON to trigger the import over 1,000 RSS feeds which need very regular refreshing.
So in short, stupid me, I was forcing Drupal to scan through 2,000 posts every 10 minutes and generate a new sitemap.xml file. You can imagine how many SQL queries that generated unnecessarily.
I disabled the module while I monitored the CPU load interactively. Almost on the spot, I could see the load on the Apache and SQL server going down. And the server has been purring happily ever since.
Since that moment, I am sitting in the corner of my room, with ashes on my head, crying “Mea Culpa, Mea Culpa, Mea Maxima Culpa”* (* “It is my fault, my fault, my freaking fault”), and hope the Server Gods won’t look badly upon me on my Digital Judgement day.
For selfhosted Content Management Systems like Drupal or WordPress alike: check each of your plugins and their settings. It takes only one plugin or one wrongly configured plugin to either slow down your site, or slow down your server.
Unless if you want to be a stupid web admin like myself, of course.