We’ve been working with Chef (formerly OpsCode) for a couple years now. I can safely say that I’ve spent more time crawling through forums and reviewing code documentation than I care to admit. In all that time, I’ve never seen anyone clearly document how to use Chef dynamically. I hope that I can share some cool ideas in this post and maybe somebody out there can make some magic happen similar to what we’ve done.
For those of you who may not know, Chef is a framework that makes system configuration ridiculously easy. Their terminology is a little strange as it follows a cooking analogy (hence Chef, get it?). A recipe is essentially a script, or some form of code that yields a result. A cookbook is a collection of recipes. Tools to interact with Chef include knife, etc. The analogy only stretches so far and soon you get into more practical terms like node, role, and environment.
I mentioned using Chef dynamically. So here’s the problem. You build a role with some recipes and assign it to a node. The node then takes the role, downloads the recipes and acts on them. The result is a configured server. It’s very cool, but what happens when you have hundreds of services iterating through hundreds of code versions, each of which configure their servers slightly differently? Managing hundreds of roles with versioning control can quickly become a nightmare. The trick is to handle all of the code dynamically and in a single maintainable place. You have one master dynamic recipe that is capable of reading in a configuration file and acting on the settings inside. Then, you take the configuration file and push it up into a Data Bag (Chef’s generalized data storage system).
For example:
The data bag:
{
“Id”:”My_stack_1_0_0_0”,
“Install_apache”: true,
“apache_settings”:{
“settting”: “blah”,
“more_settings”:”blah”
}
}
In the recipe:
### Get the data bag
my_data_bag = Chef::DataBagItem.load(“#{my_data_bag_name}”, “#{my_data_bag_version}”)
If (my_data_bag[‘install_apache’] == true)
#Do everything you need to install apache.
#and use apache_settings to do it.
End
That’s a really rough, really quick example, but it covers the gist of the idea. You can even use off-the-shelf cookbooks by taking the settings specified in the data bag and assigning them to the node as attributes, which causes them to take precedence over cookbook default values.
We’ve had the discussion about just using role files instead of data bags a number of times. Using a role is the standard for everybody out there, so why don’t we go that way? The answer is version control. Data bags can be versioned. When you’re dealing with individual specifics for each stack versioning the configuration data allows you to tie configuration directly to a version of code. This allows for dependency matching, as well as a myriad of other niceties that can be realized once you get into it. The data bags approach also allows you to easily roll back a configuration, meaning if somebody uploads a bad configuration and screws up their stack, we can just roll back to a previous data bag with no sweat off our backs. It adds a layer of complexity to the code so this isn’t an efficient answer for everything out there, but for configurations that are often changing, this makes server change control as easy as checking a file.
To make it even nicer, build a UI that creates the configuration file for you. This will allow your development teams to just click a couple boxes to indicate their desired settings, click a button, and have a pile of fully configured boxes ready to take live traffic. Just like magic.
