Posted by on January 2, 2014 in Continuous Delivery, DevOps

We’ve been working with Chef (formerly OpsCode) for a couple years now. I can safely say that I’ve spent more time crawling through forums and reviewing code documentation than I care to admit. In all that time, I’ve never seen anyone clearly document how to use Chef dynamically. I hope that I can share some cool ideas in this post and maybe somebody out there can make some magic happen similar to what we’ve done.

For those of you who may not know, Chef is a framework that makes system configuration ridiculously easy. Their terminology is a little strange as it follows a cooking analogy (hence Chef, get it?). A recipe is essentially a script, or some form of code that yields a result. A cookbook is a collection of recipes. Tools to interact with Chef include knife, etc. The analogy only stretches so far and soon you get into more practical terms like node, role, and environment.

I mentioned using Chef dynamically. So here’s the problem. You build a role with some recipes and assign it to a node. The node then takes the role, downloads the recipes and acts on them. The result is a configured server. It’s very cool, but what happens when you have hundreds of services iterating through hundreds of code versions, each of which configure their servers slightly differently? Managing hundreds of roles with versioning control can quickly become a nightmare. The trick is to handle all of the code dynamically and in a single maintainable place. You have one master dynamic recipe that is capable of reading in a configuration file and acting on the settings inside. Then, you take the configuration file and push it up into a Data Bag (Chef’s generalized data storage system).

For example:

 

The data bag:

{

“Id”:”My_stack_1_0_0_0”,

“Install_apache”: true,

“apache_settings”:{

“settting”: “blah”,

“more_settings”:”blah”

}

}

 

In the recipe:

### Get the data bag

my_data_bag = Chef::DataBagItem.load(“#{my_data_bag_name}”, “#{my_data_bag_version}”)

If (my_data_bag[‘install_apache’] == true)

#Do everything you need to install apache.

#and use apache_settings to do it.

End

 

That’s a really rough, really quick example, but it covers the gist of the idea. You can even use off-the-shelf cookbooks by taking the settings specified in the data bag and assigning them to the node as attributes, which causes them to take precedence over cookbook default values.

We’ve had the discussion about just using role files instead of data bags a number of times. Using a role is the standard for everybody out there, so why don’t we go that way? The answer is version control. Data bags can be versioned.  When you’re dealing with individual specifics for each stack versioning the configuration data allows you to tie configuration directly to a version of code. This allows for dependency matching, as well as a myriad of other niceties that can be realized once you get into it. The data bags approach also allows you to easily roll back a configuration, meaning if somebody uploads a bad configuration and screws up their stack, we can just roll back to a previous data bag with no sweat off our backs. It adds a layer of complexity to the code so this isn’t an efficient answer for everything out there, but for configurations that are often changing, this makes server change control as easy as checking a file.

To make it even nicer, build a UI that creates the configuration file for you. This will allow your development teams to just click a couple boxes to indicate their desired settings, click a button, and have a pile of fully configured boxes ready to take live traffic. Just like magic.

About Daniel Sands

Daniel Sands has been a DevOps Engineer at Ancestry.com since December of 2011. Prior to joining Ancestry.com, he served as the Vice President of Information Technologies at the Enlightened Wealth Institute based in Provo Utah from January 2008 to December of 2011. Mr. Sands holds an M.B.A. with an Information Technology specialization from Western Governers University. His favorite quote is “It’s kind of fun to do the impossible.” - Walt Disney


We really do appreciate your feedback, and ask that you please be respectful to other commenters and authors. Any abusive comments may be moderated.

Commenting is open until Thursday, 16 January 2014