Posted by Daniel Sands on May 2, 2013 in Development, Inside our Offices, Operations

For the last year and a half, we’ve been breaking in a new concept at called a DevOps engineer. There is a ton of material on the internet about what DevOps means to various groups, and how they’ve implemented it. A lot of it revolves around SCRUM, Agile processes, and other approaches to increase productivity within a team, but the underlying premises are significantly different. As the name implies, DevOps is the combination of development (dev) and operations (ops). That statement alone conjures up all kinds of interesting images. Is it a developer that occasionally racks servers? Is it a network engineer that writes hyper-intelligent scripts? Is it an installation tech that manages to automate the entire installation process? Or is it simply someone who can translate why a development team suddenly needs twice the processing power because a new version of a software platform was released with a great new feature set? There are any number of combinations that could result from joining development and operations.

As a DevOps team our vision was to provide development with an API for operations. We quickly discovered that it wasn’t necessarily how we defined ourselves, but how other groups in the organization viewed us. Long story short, development thought we were operations, and operations thought we were development. As least at first.

From the developer’s perspective, we were an ops team that was there to help developers. At first we received all kinds of interesting requests. It seemed like development hoped that we’d be the avenue to deliver all the things that operations kept promising but never got around to actively delivering. As a newly formed team, we liked the feeling of being useful and productive, and tried to facilitate as many requests as possible, but quickly found ourselves overwhelmed. After all, if an entire ops organization couldn’t manage to accomplish the laundry list of developer requests, why would a single team, that was still getting its bearings, be able to?

Once we realized that we were getting nowhere quickly, we decided to try facilitating the needs of the developers instead of the actual requests. That isn’t to say that the developers didn’t know what they wanted, but rather, the developers may not have been aware of other avenues that could make life easier for everyone. The number one complaint from developers was that turn-around times for hardware were way too long. Lowering server delivery times was going to be an easy win, as the company had already begun experimenting with virtualization, and it was a short leap to go from experimentation to implementation. In the matter of a few months, we managed to drop the time for server delivery from months to days. Ops was able to monitor the virtual host load to know when more hardware was required, and order proactively. Developers got their servers much faster, and productivity increased as a result. Win-Win right? Except that DevOps suddenly became the man in the middle for every server configuration situation imaginable.

That kicked off our journey to create self-service tools to do all the manual yet very automatable processes. Everything from VM creation and configuration to deploying code and monitoring. These features, while independently insignificant, could add up to a ton of time that the developers didn’t want to spend on operational concerns. They wanted to develop and we were more than happy to facilitate. Once they realized that we were working in their best interest they were more than happy to switch over to our automated processes. That freed them up to focus on development and furthering the business goals.

From operations perspective, we initially were another demanding dev team that wanted more of their precious resources, which were often already spread thin. Coordinating with several different operations teams, each with their own field of responsibility, is a significant undertaking for any team. Initially the DevOps team would get resistance from various ops teams that felt we had no business trying to do their jobs when they were perfectly capable of doing it themselves. But we didn’t want to do their jobs for them. We wanted to give them tools to make their jobs easier. Why manually create 50 active directory objects, each with a slew of details, when a script could easily handle it for you? Why bother tracking static IP address allocation by hand when it could all be done in a database with a nice API? It took a while to find common ground, but eventually many of the time intensive tasks that consumed much of ops’ precious resources were scripted, allowing them to focus on improving infrastructure, and making the site run better.

As I said before, the team has been around for over a year now. Our vision was, and still is, to provide development with an API for operations. It will be pretty cool one day for a developer to be able to poke a single API and get everything they need to get their code from inception to delivery. On the operations side, it would be pretty cool if all the automatable minutia was all handled automatically, so they could completely focus on improvements. We’re getting there, and quickly.

Daniel Sands

Daniel Sands has been a DevOps Engineer at since December of 2011. Prior to joining, he served as the Vice President of Information Technologies at the Enlightened Wealth Institute based in Provo Utah from January 2008 to December of 2011. Mr. Sands holds an M.B.A. with an Information Technology specialization from Western Governers University. His favorite quote is “It’s kind of fun to do the impossible.” - Walt Disney