Tech Roots » Harold Madsen http://blogs.ancestry.com/techroots Ancestry.com Tech Roots Blogs Wed, 29 Apr 2015 16:23:51 +0000 en-US hourly 1 http://wordpress.org/?v=3.5.2 External APIs: To Explode, or Not to Explode, That is the Questionhttp://blogs.ancestry.com/techroots/external-apis-to-explode-or-not-to-explode-that-is-the-question/ http://blogs.ancestry.com/techroots/external-apis-to-explode-or-not-to-explode-that-is-the-question/#comments Mon, 29 Sep 2014 17:00:30 +0000 Harold Madsen http://blogs.ancestry.com/techroots/?p=2830 Shakespeare might not approve of my taking liberties with his play Hamlet, though prince Hamlet was essentially saying the same thing as I was feeling last year: To be, or not to be, that is the question— Whether ’tis Nobler in the mind to suffer The Slings and Arrows of outrageous Fortune, Or to take… Read more

The post External APIs: To Explode, or Not to Explode, That is the Question appeared first on Tech Roots.

]]>
William Shakespeare - hs-augsburg.de

William Shakespeare – hs-augsburg.de

Shakespeare might not approve of my taking liberties with his play Hamlet, though prince Hamlet was essentially saying the same thing as I was feeling last year:

To be, or not to be, that is the question—

Whether ’tis Nobler in the mind to suffer

The Slings and Arrows of outrageous Fortune,

Or to take Arms against a Sea of troubles…

Would Hamlet go on or cease from all? Yes, I may have felt just as Hamlet in the Nunnery Scene when I thought about my “sea of troubles” just one year ago. Well, maybe I’m waxing a bit too dramatic but there were real concerns on my part regarding last year’s events (oh how “smart a lash” that memory doth make!). What was that worrisome memory? Allow me now to retrace my steps to that challenging day and give you context to my soliloquy.

This story begins last fall and there are many actors on the stage of events. Yes, my story begins with our mobile app and our external API and reaches its climax when seemingly all users download their family trees all at once! Oh the misery. Let us begin our tale of woe.

Ancestry.com (the bastion of family history) has an external API that is used by our mobile apps and other strategic initiatives to share and update a person’s family tree, events, stories and photos. Our external API has been most important in our mobile efforts (11 million downloads) and in working with the popular TV series, “Who Do You Think You Are?” Our mobile team had successfully grown our mobile usage to such an extent that I began to worry it might actually tax our systems. That concern was beginning to bubble up from my subconscious the fall of last year and this leads us to that disquieting day. Last year, our iOS mobile app was promoted in the Apple App Store because of the updates we had made for the release (along with other Ancestry promotions). Those promotions led to large numbers of simultaneous family-tree downloads and it weakened the mighty knees of our backend services. We endured a week of utmost traffic and were consigned to throttle (limit usage of) our mobile users (the API team saved the day by throttling usage and thus preserving the backend services). After experiencing that calamitous week, we might well have cried, “Get thee to a nunnery!” or “O, woe is me” but we repressed such impulses.

OK, it wasn’t actually a “calamitous week”, I was just getting into my Hamlet character. Given that impact to our website was quite minimal, most of our users had a good experience. However, it was a bit frustrating for many of our mobile users – it took too long for many to successfully download their family tree to their mobile device. This really is great news that our mobile traffic has been growing. We realized that we must architect a plan to take us through the next round of application and user growth. Here’s how it happened:

That experience caused us to reconsider how we deploy our mobile apps, how our mobile apps interact with the API, how we call our backend services, how we deliver a tree download and if we should continue to aggregate our services at the API layer. Each of these areas of the company went under a review to see how we might mobile to api to servicesoptimize our systems. After holding periodic meetings, discussions and code reviews over several months a plan began to gel. Below is a list of some changes made to our systems and application:

  • Pass Through: Rather than aggregate our services at the API layer, we took the strategy of creating a “pass-through” layer back to our backend services. This put the responsibility directly on our services to further optimize their code, and in some cases, create new endpoints specifically with mobile usage in mind. This methodology also enabled our mobile teams to more effectively cache data according to their needs and Service team recommendations. More on that below.
  • Mobile Usage: As our users became more mobile we have increased traffic through our APIs from mobile devices. Last fall our mobile usage at Ancestry.com reached critical mass and put serious pressure on our services (especially during big promotions and app updates). Because mobile usage differs from the website in important ways it was time to address this in our backend services. After several meetings involving cross-functional teams, a few service calls were designed with mobile usage in mind. One of the results was that downloading your entire family tree became much faster. Downloading a tree with 10 thousand persons (and all their associated events etc.) decreased in time from several minutes to under 1 minute.
  • Cashing: Because we changed our API model to a pass-through, our mobile app could now cache data from each call at appropriate intervals thus taking pressure off of our backend services and network. This meant fewer calls (in the long run) to the external API.
  • Mobile App Optimization: One area of review was our mobile application. After the code review we theorized that our app might have put undue pressure on our services. What was the root cause? Apple has two new, interesting features:
    • Apple can automatically download and install new applications on your iPhone or iPad
    • Apple can wake up apps in the background and do tasks

When we released our app last year, we believe it was automatically downloaded by Apple (onto most Apple devices) and then, in the background, automatically downloading that user’s main tree. To be sure, this process would have happened anyway once the end-user opened the app manually (that was required for that app update) but doing it manually would have helped spread the traffic over several days rather than all at once. Of course this is just theory but we wanted to ensure it was not happening and would not happen next time.

  • User Queuing: As you know, queue is just another word for getting in a line. People get in line to buy a new iPhone or to buy tickets for a concert. That’s what we do when there are too many requests at a given moment. Anticipating high traffic from our new 6.0 mobile app (plus other site promotions at that time of year), we created a new way of throttling too-high traffic. Rather than throttling a percentage of calls to our API (making it hard for any-one user to successfully download their family tree to their mobile device) we created a system called User Queuing which allowed a certain # of users into our system at one time. By allowing X number of users into our systems for 10 minutes of uninterrupted usage ensured each would have a pristine experience. This would also protect our backend services from being overloaded as well. We could adjust on the fly how many users were allowed through our API at any one moment. Thus more individuals would have a better experience and the others would be invited to return in a few minutes. We would only turn-on User Queuing if too many users made requests at the same moment.
  • Load Tests: To ensure our systems and new service calls could handle beyond-expected peak calls we ran them through a gauntlet of load tests. These series of tests ensured we had proper capacity.

Now, once our app was approved by Apple, we could have immediately released our app but there were things to consider. Here is how we timed the successful release:

  • We received permission from Apple to release our app in the app store the day before the Apple promotion – thus helping us take some of the steam off of the release.
  • We decided to release at a time of day when we anticipated traffic would be somewhat low.
  • We decided to release when our engineers and database administrators were all available in case we needed to react quickly and also to monitor traffic.

Finally, the day arrived and we were ready. All hands on deck. User Queuing ready to trigger. There was great excitement and nerves. How would our systems hold out? Which internal system might buckle under pressure or show up with a previously undiscovered bug? How long after the launch would we need to kick in User Queuing and how many users would be temporarily turned away by the queue? Did we have enough servers, or memory or database throughput? On the other hand, we had tested our code so well, how could it fail? There was much excitement in the air.

All engineers were readied…and…the button was pushed to release our new mobile app!

Did it all collapse? Were there cascading failures? Was the load too much to bear? Did everything explode?

Nope!

Nothing happened, OK, it seemed like nothing. The load gradually increased over the next few hours but our systems held up wonderfully. No strain, no collapse, no running low on memory, no bottlenecks. Nothing. Yes, there were a few minor bugs to fix but most customers had a great experience and it went very smoothly. team lunchThe time, effort, and planning paid off. It worked!

We were so happy – and relieved. We had done our job. In the coming days several teams went to lunch to celebrate the successful execution of months of planning and work. Some of the engineers actually smiled on the day that nothing happened. Even Hamlet dropped by and asked me a question: “Didst thou not explode with a sea of troubles?” And I said, “not on your life!”

The post External APIs: To Explode, or Not to Explode, That is the Question appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/external-apis-to-explode-or-not-to-explode-that-is-the-question/feed/ 0
Featured Article: Want Great APIs? Start With Traininghttp://blogs.ancestry.com/techroots/featured-article-want-great-apis-start-with-training/ http://blogs.ancestry.com/techroots/featured-article-want-great-apis-start-with-training/#comments Tue, 03 Jun 2014 15:59:47 +0000 Harold Madsen http://blogs.ancestry.com/techroots/?p=2443 Ancestry.com, has awesome software engineers, products, and APIs. However, programmers are not always trained as API designers and when it comes to API development, consistency matters. As companies build their API programs using multiple teams, APIs tend to develop their own personalities and become radically different from one another. That’s a problem. Fortunately, it doesn’t… Read more

The post Featured Article: Want Great APIs? Start With Training appeared first on Tech Roots.

]]>
Ancestry.com, has awesome software engineers, products, and APIs. However, programmers are not always trained as API designers and when it comes to API development, consistency matters. As companies build their API programs using multiple teams, APIs tend to develop their own personalities and become radically different from one another. That’s a problem.

Fortunately, it doesn’t have to be that way. Companies can get consistency in their APIs through development standards as well as engineer training. If all developers adhere to one set of guidelines and standards, all your APIs will feel similar.

“What’s the benefit of that?” you might ask. “Why should I take the trouble to make similar APIs throughout the company? That’s a lot of work, time, and coordination.”

Great questions! The lessons learned from our efforts in creating clean, easy to use, and accessible APIs have been featured in the InformationWeek Strategic CIO section online. You can view our full story here.

The post Featured Article: Want Great APIs? Start With Training appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/featured-article-want-great-apis-start-with-training/feed/ 0
APIs Are Like Parentinghttp://blogs.ancestry.com/techroots/apis-are-like-parenting/ http://blogs.ancestry.com/techroots/apis-are-like-parenting/#comments Mon, 03 Mar 2014 15:00:40 +0000 Harold Madsen http://blogs.ancestry.com/techroots/?p=1995 I’ve presented at several conferences recently and one of the analogies that resonated with the audience was that of comparing API Design to parenting.  So, here’s the analogy: APIs Are Like Parenting… The year was 1966. My family was living in Ethiopia while my dad taught at the American university as a guest professor. I… Read more

The post APIs Are Like Parenting appeared first on Tech Roots.

]]>
I’ve presented at several conferences recently and one of the analogies that resonated with the audience was that of comparing API Design to parenting.  So, here’s the analogy: APIs Are Like Parenting…

The year was 1966. My family was living in Ethiopia while my dad taught at the American university as a guest professor. I awoke one morning to hear my dad rousing the family at 6am for “Family Morning”. My dad would walk through the house singing his wakeup song and gathering the family to read children stories and scriptures. Then we would say family-morning prayer and enjoy my mom’s hot breakfast. As a young child, I have to admit it was pretty annoying to wake up so early in the morning, however, looking back now it is a cherished memory.

Family in Africa

Several pictures of me and my family in Africa

It wasn’t just the exotic world of Africa with servants, a pet baboon and visits to the Tisisat Falls (Blue Nile), for example, but the enjoyment of family activities too. In fact, I think these family events, along with other family traditions, are what makes our family so tightknit. Even after 40+ years since our 4-year African adventure, we still enjoy hanging out together.

My family experiences have had a huge impact on my life. My parents were not perfect, but were kind, gentle and incredibly consistent. In my mind, that consistency makes a big difference in the success of a family. Recently I was preparing a presentation on APIs when it occurred to me that designing successful APIs has surprising commonality with raising successful familieshang with me here and let me explain (yeah, call me a nerd – it’s true).

Mobile app using Ancestry API

Mobile app using our API

Like most technology companies, Ancestry.com has many APIs. What’s an API you ask? Well, in a nutshell, it’s what programmers use to create a computer application. For example, Ancestry has some really awesome mobile applications. These mobile apps show your family tree with relationships, events and photos. How does that app get your family tree from Ancestry? It uses the Ancestry API to programmatically and securely access and update your family tree. It’s pretty cool stuff.

Doors with bad design

www.baddesigns.com

Developers create APIs of all kinds, and depending on their design skills, the API can turn into something easy to use or difficult to use. Let me digress and explain this a little further. There is a concept called Design Affordance. Design Affordance says that the way an item (such as a door or a filing cabinet) is designed communicates how it should be used. The design of the item speaks louder than the documentation. Take these doors in the picture to the right for example. The design of the door is quite poor. It’s so poor that instructions were placed on the door to help the user know what to do. The design can be good and helpful or poor and a hindrance. At Ancestry we promote good API designs so they are intuitive and easy to use. That way we can more quickly build products for our customers.

So, how are APIs like families? I promise, I’m getting there. Just a few more words about families to properly demonstrate the analogy. I’m not a parenting expert but I think I have a pretty good nose for the topic now that I have raised several children of my own. Parenting is so rewarding! And challenging too! It’s just a lot of work mixed with fun and rewards. But now that my kids are mostly grown the rewards are rolling in big time.

Family picture

My family

I love hanging out with my adult children (half are married now). They are such great people and terrific friends. We have so much fun together. We look for any excuse to have a party with food and fun and just be together. But getting to this point took a lot of work, love and consistency.

As I look around at other families that also have success, I notice they have different parenting styles from me and yet their children turn out fine too. So is there any connection between successful parents? Maybe. I think the connection between most successful families boils down to love and consistency. If you spend time with your children, enjoy them, love them, have patience and are consistent with your rules, punishments and rewards then it usually works out. Certainly there are no guarantees but these things help.
So now the tie in. Successful APIs are like successful families in the sense of attention and consistency. It takes a lot of love (attention) and consistency to create an API that is easy and fun to use.

Princess Bride - Let me explain

Princess Bride (www.shanelien.com)

I’ve got a presentation that explains how to design awesome APIs but I’d need an hour. But as Inigo Montoya said in the movie Princess Bride, “Let me ‘splain. No, there is too much. Let me sum up.”  OK, “let me sum up” a few things that make for a great API:

  • Consistent naming conventions
  • Standard terminology
  • Uniform error responses
  • Attention to detail (that’s the love part for APIs)
  • REST APIs that work with only 2 resources and use 4 HTTP verbs (oh, there is so much more to talk about here…)
  • Avoid API design by way of method-driven approach – that leads you down a slippery slope

It’s really hard to sum up good API design in a short blog, but the bottom line is to be consistent. Stop and pay attention to the API designs, make sure they are consistent throughout and take pride in the results and in good, discoverable documentation. If you do, then developers will thank you for caring and your API is more likely to be used. They are more likely to wake up in the morning and rather than say, “Oh no! I have to go to work and use THAT API,”, they will say, “Yes! I get to go to work use that awesome API!”

What successes or challenges have you had with good or bad API designs?  Leave a comment and let me know.

 

The post APIs Are Like Parenting appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/apis-are-like-parenting/feed/ 2
Ancestry.com Gets Hands-On With API Design at RootsTech Innovators Summithttp://blogs.ancestry.com/techroots/ancestry-com-gets-hands-on-with-api-design-at-rootstech-innovators-summit/ http://blogs.ancestry.com/techroots/ancestry-com-gets-hands-on-with-api-design-at-rootstech-innovators-summit/#comments Mon, 03 Feb 2014 05:00:15 +0000 Harold Madsen http://blogs.ancestry.com/techroots/?p=1812 When it comes to API interface design, do your software engineers have a design touch or feel? Many engineers do not. Without training and proper aesthetic feel, your APIs might end up quite awkward or even a bit messy. Good API design is something that most engineers are not born with but with proper training,… Read more

The post Ancestry.com Gets Hands-On With API Design at RootsTech Innovators Summit appeared first on Tech Roots.

]]>
When it comes to API interface design, do your software engineers have a design touch or feel? Many engineers do not. Without training and proper aesthetic feel, your APIs might end up quite awkward or even a bit messy. Good API design is something that most engineers are not born with but with proper training, engineers can avoid the slippery slope to bad designs.
If you want to improve your API design capabilities, join me for some fun, hands-on experience at RootsTech 2014. This Wednesday, I will provide hands-on design training at the RootsTech Innovator Summit.

Session info:
Wednesday, February 5, 2014, 2:00 PM (room 251A)
Presented by Harold Madsen (Ancestry.com)
At the East Wing of the Salt Palace Convention Center, SLC, UT

The post Ancestry.com Gets Hands-On With API Design at RootsTech Innovators Summit appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/ancestry-com-gets-hands-on-with-api-design-at-rootstech-innovators-summit/feed/ 0
Ancestry.com to present at API Strategy & Practice Conferencehttp://blogs.ancestry.com/techroots/ancestry-com-to-present-at-api-strategy-practice-conference/ http://blogs.ancestry.com/techroots/ancestry-com-to-present-at-api-strategy-practice-conference/#comments Mon, 21 Oct 2013 06:00:43 +0000 Harold Madsen http://blogs.ancestry.com/techroots/?p=1314 Designing easy-to-use APIs is not always an easy thing to do.  In fact, some developers feel guilty taking time from coding for design – especially spending design time on such things as method names, request and response objects or service names.  If you don’t take the time upfront then you are destined to deal with… Read more

The post Ancestry.com to present at API Strategy & Practice Conference appeared first on Tech Roots.

]]>
Designing easy-to-use APIs is not always an easy thing to do.  In fact, some developers feel guilty taking time from coding for design – especially spending design time on such things as method names, request and response objects or service names.  If you don’t take the time upfront then you are destined to deal with technical debt (design debt) later.  How would you feel going to work each day knowing you will use an API that was poorly designed?  Yeah, not good.  This Thursday, I’ll be speaking at the API Strategy conference, presenting a case study describing how Ancestry.com reinvented its internal APIs to be more consistent, discoverable and easy to use – while simultaneously serving millions of subscribers.

Session info:

Thursday, October 24, 2013, 11:25 AM session, track 2

Presented by Harold Madsen (Ancestry.com)

At the San Francisco Parc55 Hotel

 

*Update: You can view the presentation below.

 

 
 

The post Ancestry.com to present at API Strategy & Practice Conference appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/ancestry-com-to-present-at-api-strategy-practice-conference/feed/ 0