Lumps of Metal

I’ve just finished another article for Linux Magazine on good server design for the cloud.

Cloud computing is amazing, everyone is talking about it and it will take over the world. One day it will actually be possible to use the cloud to convert your DNA into a Microsoft Tag. Maybe.

But it’s really easy to use cloud computing badly because we’ve all been brought up on traditional servers: lumps of metal. This misses the point. You end up with a badly configured cloud: some scaling is easy but big configuration changes are scary. You don’t make them because they’ll be time consuming and risky. Not very elastic?

With a well configured cloud setup you can start and stop servers automatically. You can create a complete clone of your live environment without worrying about breaking things. You can roll-out configuration changes in a controlled manner and revert the entire farm if things don’t go to plan.

In the article, I’ve covered tools such as Scalr, which we are using to manage our elastic hosting, and RightScale which is also worth a good look. These manage the ‘farm’ – how many servers you have, what they do and how to scale when (for good reasons) you’re featured on the 10 o’clock news.

Getting an app or a website running on the cloud properly isn’t just a matter of copying it across. It doesn’t take much, but making sure your app has no bottle-necks and avoiding some bad (but normal!) practices of storing stuff on the filesystem turn a bulky, un-scalable app into a lovely, scalable, cloud-friendly app.

Even configuring servers works slightly differently. As Andrei pointed out, cloud images (which are like “ready to go servers”) are very heavy, clumsy ways of designing your cloud. The image should contain JeOS (Just enough operating system) which for us is ubuntu plus a few other bits. You then use tools such as Scalr to install and configure the software you need.

What’s remarkable is that the amount of code and scripting needed to get a scalable cloud farm up and running is not that huge. If you avoid spinning your own management tool and use Scalr, RightScale or something similar it is possible to get something going within a few hours.

The next article I’m working in is about putting all the code which is used to configure the servers into a debian package, which means it can be installed very, very easily on Ubuntu and turn any old server into part of our cloud.

Written by dan on the 4th May 2010