We rolled out some smaller features and bug fixes recently, here's the quick summary.

To both the cloud view and an instance's detail page we added an easy way to copy the internal and external IP and host names to your clipboard. It's not hard to overlook, but here's a couple of screenshots to get you going. The cloud detail page includes buttons to copy the private and public IP address:

And the same on the instance detail page:

Pretty neat, if you need to copy around IP addresses to set up DNS names, or for use in your Chef cookbooks. The latter is not exactly recommended, we encourage on the data that's delivered to an instance during a Chef run, but from time to time you just gotta do what you gotta do.

In other feature news, it's now possible to re-assign an Elastic IP from one instance to another. Say you want to do a seamless failover from a smaller load balancer instance to a bigger one, or move your full setup from one cloud to another without having to change DNS names, that's now possible. You can reassign an Elastic IP to any instance managed by Scalarium, not matter if it's running or not.

Of course that will mean that the instance it belonged to will be assigned a new IP address by EC2. That may take a while, even up to a couple of minutes, we've seen it take two to three minutes on average. Assigning the Elastic IP to a new and running instance however, doesn't take more than a couple of seconds. It just takes Amazon a while to assign a new and random IP to the instance it belonged to. So the switch is almost immediate for the new instance, but it may take a while for the old instance to get a new IP.

Don't forget though that Elastic IPs assigned to instances that aren't running will cost $0.01 per hour. Elastic IPs are only free when they're used by an instance.

While we're at it, Elastic IPs so far weren't automatically assigned to instances that existed before you enabled "Auto-Assign Elastic IPs" for a role. That oddity is now also fixed. So when you start an instance for a role that has this option enabled, we'll get an Elastic IP for it, no worries.

We also extended the Ganglia statistics to include much more detail e.g. on Passenger (for Rails applications) and Apache requests, including statistics on what kinds of status codes are returned, how much memory Passenger processes use, and how many of them are running. Monitoring? We've got you covered!

Until now, we had some problems (that we didn't know of) handling instances in the US-East region. Our thanks to Till for helping us finding them and figure out what's wrong. Turns out the US-East region is quite a finnicky beast, having some oddities and very long delays in their APIs compared to all the other regions. For instances in that region, we can't show you the private IP within Scalarium, because the private DNS name doesn't follow a scheme that includes the IP in the name, only the internal MAC address, which isn't very helpful. You can still find out the internal IP in your Chef cookbooks using node[:ipaddress], and it's safe to just use the host name in most cases.

On a smaller side note, did we mention we're not private beta anymore? No need to register for an invitation, go straight to signing up, like, now!

Even though it's Friday we're far from done introducing our new features. I'm going to focus on smaller but still rather interesting ones this time.

EBS Snapshots

The beauty of EBS is that you can take snapshots at any time. Once you taken one, all subsequent snapshots will store only the differences from the last snapshot, block by block. A very useful feature to get regular backups for your database server. We've added a way to get manual snapshots in Scalarium. Don't worry, automating them for you is already in the works.

You can find the details on every volume in your account either on the instance it's attached to, or on our "Assets" page. You should go ahead and create a snapshot right away.

We're showing you the progress as we get updated data from Amazon's API. Taking a snapshot can take a while depending on the the volume's size.

You can take as many snapshots as you like, but be aware that by default, your Amazon Web Services account can only have 500 EBS snapshots all together. You can contact Amazon to increase that number.

Now, the fun starts when you create a new volume. You can select existing snapshots that should be used to create the volume from. If you have a working backup snapshot of your database server, and want to create a new volume for your staging cloud, based on production data, that's how you can do it. No dumping and loading the database manually required.

Custom JSON

I already introduced "Custom JSON" in yesterday's tutorial on customizing your Scalarium Cloud using Chef. Be sure to check out the tutorial, but let's look at what custom JSON means. Every time we run Chef on an instance we send it a bunch of configuration information, wrapped in JSON.

So far the only way to customize the attributes was to write your own cookbooks, specifying the attributes in them. That falls apart though when you want to reuse cookbooks on different clouds, having different attributes. Say you want your MySQL server to have different buffer settings for your testing cloud, you had to have different cookbooks for both cases.

We also got regular requests from customers wanting to overwrite our default configuration settings. The answer is pretty simple, and it's called Custom JSON. You can specify a custom JSON string for every cloud within Scalarium. We'll merge whatever you specify into the configuration that's generated for each instance. So you're free to overwrite our settings, or to simply overwrite settings in your own cookbooks, or even just to overwrite the defaults in our example cookbooks.

You can find the setting on every cloud, by following the "Edit Cloud" link in the "Actions" menu.

Now you can specify a valid JSON string (we'll validate it before you save the changes, don't worry). I'm overwriting the cookbook's defaults for the maximum amount Redis' virtual memory system should use.

All subsequent events sent to the instances in this cloud will have the custom JSON configuration merged into them. Neat!

Freeze/Unfreeze Instance Configuration

Scalarium constantly manages the lifecycle of your cloud. Whenever an instance comes or goes, Scalarium notifies the rest to update their configurations accordingly, but only if necessary. Did you know that can utilize this event to write custom configurations?

But sometimes you want to just leave an instance be once it's fully set up, not having to bother with changes in its cloud. Scalarium sports a new feature to do just that. For every role you can now freeze and unfreeze the configuration for all instances in it.

Unfreezing a role will unfreeze all instances in it, even if you just froze one. Which you can also do on every instance's detail page.

Manually Trigger Lifecycle Events

Sometimes you just want to re-run the setup on an instance, say when you updated your custom cookbooks, or manually trigger the configure event to update configurations and what not. No worries, we've got you covered. On every instance's detail page you can trigger both setup and configure manually.

Note that you can trigger the configure event for instances with frozen configuration. Re-running the setup requires your Chef cookbooks to be idempotent, because it will re-run the whole setup procedure. If packages are already installed, Chef won't install them, if configuration files haven't changed, Chef won't update them. But if you for example download source packages and build them using Chef, circumventing the package system (a viable choice sometimes), be sure to include checks that those parts of the recipes don't re-run when the software has already been installed.

Disable Auto-Healing

Auto-healing is a neat feature. Should your instance become unresponsive, Scalarium will take notice and shut it down after a grace period, bringing up a new one automatically. It doesn't happen a lot, but sometimes EC2 has a glitch that can affect your instance. Don't worry, you usually won't even notice, as we're taking care of that automatically. But sometimes you just don't want that, and maybe dig for the root cause of the problem yourself. For that, you can now disable auto-healing for every role in a cloud, in a new tab "Role Settings". It's enabled by default for all roles and instances.

That's it for this week's feature showcase. Please don't hesitate to get in touch if you have any questions.

It's no secret that we're using Chef to manage your infrastructure on Scalarium. It allows us to write system management descriptions in the way we're most comfortable with, and it allows you to fully customize and adapt your infrastructure to your needs. The thing about the combination of Scalarium and Chef is: It's not always easy to get a good grasp on what you can do, and how to best approach to setup customized components and services. We're getting a lot of questions on this, and it's time to offer a in-depth and step-by-step tutorial to show you how you can set up your own services using Scalarium and Chef.

We're covering all the bases, Chef basics, how you can hook into the Scalarium lifecycle event, and how you can utilize both to easily set up new services and automatically reconfigure your cloud when something changes. So without much further ado:

Customizing your Scalarium Cloud using Chef

Please let us know what you think, and do not hesitate to get in touch if you still have questions. We'll be updating this tutorial as we find that things are still missing, and we'll publish a more tutorials and new screencasts in the upcoming weeks.

One of the really neat things about the elasticity of the cloud is the fact that you can spin up new instances at any time and shut them down at a later point in time. If you need additional power just for a specific time frame, say business hours, and you know upfront on what day of the week you need them, you can boot them up easily with Scalarium.

But wouldn't it be neat if your cloud management tool could do the job for you? Yeah, we think so too, it totally would.

Good news! We rolled out time-based auto-scaling recently. Now you can tell us when you need additional computing power, and how much you want, and we'll take care of it for you. The time frame is specified in full hours, so if you need an additional instance from just 8 to 9 in the morning, no problem. You can specify for any day of the week and for every instance seperately.

But I'm getting a little ahead of myself here, let's look at how it's done. When adding a new instance to a role, you'll notice that we've split it up into three parts, and that there's a new tab called "Time based Instance".

For time-based auto-scaling we don't use your existing instances, we allow you define a separate stash of instances whose only purpose is to serve additional computing power, leaving your existing instances (which you control manually) untouched.

So if we go ahead and create a new time-based instance using the form hidden behind the new tab we can give it a spin. The new instance is put into a different group and sports a tiny icon to make it recognizable as an instance that's only active during specific hours on specific days.

When you created the new instance, we'll immediately take you to a new dialog, where you can specify the hours and days of the week the instance is supposed to run for every time-based instance in a specific role.

The list is sorted by availability zone, listing all the time-based instances in each zone. You can specify times for every day of the week, or only for specific days. Every box describes one hour in the day. If an instance has only a couple of days in the week configured, we make that visible by giving the box a lighter green on the "All" tab, as you can see on the last entry "mcfly".

And that's all there's to it. Neat! Now, if you specify an hour that falls into the current time of day (all times are UTC), the instance will be booted up automatically in the next couple of minutes. We don't just check every hour, we constantly check for updated auto-scaling settings. That's also true for disabling auto-scaling. So if you decide you no longer need to run a time-based instance, you can just unmark the corresponding hours, and we'll shut it down again within minutes.

Of course the above image showing the instance booting could be totally photoshopped, but you'll have to take my word for it, that it was just started up automatically. But hey, why not try it yourself?

So there, need processing power only during the day? Just tell us the time frame, and we'll take care of it for you. Time-based auto-scaling is just on of the features we've recently rolled out. Stay tuned!

One of the big requests we've been getting was built-in monitoring. We already offer a certain amount of monitoring built-in, but it's not sufficient for a really big picture. With Chef cookbooks it's easy to set up your own monitoring using e.g. Munin, Ganglia, Nagios and the like. But since it was an often requested feature we decided to build it right into Scalarium to make integration into your clouds simple and seamless, with new nodes being added to the monitoring setup automatically.

Setting Ganglia up for your cloud is simple. Just add a new role to your cloud and select the "Ganglia Monitoring Master".

There's a couple of settings available. The Ganglia server will install Apache, and you can specify a mount point URL under which you'd like the Ganglia statistics to be available, and a password to protect the monitoring data from unauthorized access. The default username is scalarium, but can be changed in the Ganglia Settings once you've added the role. You can change username and password even after the instance was started, it'll be automatically reconfigured during the next deployment, or when the state of your cloud changes.

Add an instance to the role, and boot it up. New instances will now automatically be configured to report data to the master instance. There's a slight restriction for now: The server running Ganglia can't serve any other web applications. We're working on a fix for that, but you can simply promote e.g. your database server to be responsible for collection statistics.

Once setup is complete, and you've done at least one full deploy (if all of your instances were stopped before, the full state of your cloud wasn't known yet to the instances), the Ganglia statistics are available on the instance with the specified sub-URL (e.g. http://79.125.74.221/ganglia), where you can log in using the configured credentials.

Once you've logged in, you'll be greeted with something like this, a neat overview of CPU, memory and disk usage, and some statistics specific to instances of certain roles.

Sure, it's not a beauty, and we sure wish we could offer you something as good-looking as the Scalarium user interface, but it's still incredibly useful to get a single overview on what's happening in your cloud. We'll try to improve on the default layout for the Ganglia pages in the future, but really, it's more about getting an overview of the data and your cloud's status than getting eye candy.

Each role gets a specific set of Ganglia statistics, so you can see MySQL statistics, details on the HAProxy queue, or the size of your Memcached server. By choosing a node in that handy select box above the statistics you can get even more glorious statistical detail on a specific node.

Automated Ganglia setup is just one feature we've rolled out during the last weeks. Stay tuned for more!