Tech Roots » Analytics http://blogs.ancestry.com/techroots Ancestry.com Tech Roots Blogs Wed, 20 Aug 2014 15:34:47 +0000 en-US hourly 1 http://wordpress.org/?v=3.5.2 DNA and the Masses: The Science and Technology Behind Discovering Who You Really Arehttp://blogs.ancestry.com/techroots/dna-and-the-masses-the-science-and-technology-behind-discovering-who-you-really-are/ http://blogs.ancestry.com/techroots/dna-and-the-masses-the-science-and-technology-behind-discovering-who-you-really-are/#comments Wed, 12 Mar 2014 19:02:58 +0000 Melissa Garrett http://blogs.ancestry.com/techroots/?p=2075 Originally published on Wired Innovation Insights, 3-12-14. There is a growing interest among mainstream consumers to learn more about who they are and where they came from. The good news is that DNA tests are no longer reserved for large medical research teams or plot lines in CSI. Now, the popularity of direct-to-consumer (DTC) DNA tests… Read more

The post DNA and the Masses: The Science and Technology Behind Discovering Who You Really Are appeared first on Tech Roots.

]]>
Originally published on Wired Innovation Insights, 3-12-14.

There is a growing interest among mainstream consumers to learn more about who they are and where they came from. The good news is that DNA tests are no longer reserved for large medical research teams or plot lines in CSI. Now, the popularity of direct-to-consumer (DTC) DNA tests is making self-discovery a reality, and is leading individuals to learn more about their genetic ethnicity and family history. My personal journey has led to discoveries about my family history outside of the United States. On a census questionnaire I am White or maybe Hispanic. My genetics, however, show I am Southern European, Middle Eastern, Native American, Northern African, and West African. And who knew that DNA would connect me with several cousins that have family living just 20 miles of where my mom was born in central Cuba?

Major strides have been made in recent years to better understand and more efficiently analyze DNA. Where are we today?

  • Easier: DNA testing required a blood draw. Now, you can spit in a tube in the comfort (and privacy) of your own home.
  • Cheaper: In 2000, it took about 15 years and $3 billion to sequence the genome of one person. Today you could get your genome sequenced for a few thousand dollars. To put that into context, if a tank of gas could get you from New York to Boston in 2000, and fuel efficiency had improved at the same pace as DNA sequencing, today you could travel to Mars (the planet) and back on the same tank of gas.
  • Faster: Companies of all kinds are quickly innovating to keep up with demand and to make DNA testing more readily available and affordable. Illumina recently announced a whole-genome sequencing machine that could sequence 20,000 entire genomes per year.
  • More information: We can now tell you things about your ethnicity, find distant cousins, tell you whether a drug is likely to benefit or harm you, and tell your risk of diseases like breast and colon cancer.

It isn’t all roses. There is a joke among the genetic community that you can get your DNA sequenced for $1,000, but it will cost $1,000,000 to interpret it. DNA is complex. Each of us contains six billion nucleotides that are arranged like letters in a book that tell a unique story. And while scientists have deciphered the alphabet that makes up the billions of letters of our genome, we know woefully little about its vocabulary, grammar and syntax. The problem is that if you want to learn how to read, you need books, lots of them, and up until recently we had very few books to learn from.

To illustrate how complex it can be, let’s look at how to determine a person’s genetic ethnic background. Say you are given three books written in English, Chinese and Arabic. Even if you don’t speak the languages you can use the letters in those books to determine what percent of a fourth book is written in each of the respective languages, since those three languages are so distinct. But that is like determining whether someone is African, White or Asian, which doesn’t require a genetic test. What if the three books were written in English, French and German that use a similar alphabet? That is like telling someone that is White that they are a mix of various ethnic groups. That is a much harder problem and one that usually requires a genetic test.

So how do we distinguish the different ethnicities using DNA? Since we don’t have a genetic dictionary that tells us what we are looking for, scientists use the genetic signatures of people who have a long history in a specific region, religion, language, or otherwise practiced a single culture as a dictionary. Once enough of those genetic sequences are gathered, teams of geneticists and statisticians use the dictionary to define what part of your genome came from similar regions.

How does big data play into all of this science?

DNA has been “big data” before the term became popularized. The real question should not be about how much data you have, but what you do with the data. Big data allows companies like Ancestry.com to compare 700,000 DNA letters for a single individual against the 700,000 DNA letters of several hundred thousand other test takers to find genetic cousins. That’s a lot of computational power, and the problem grows exponentially. To make all of this possible, big data and statistical analytics tools, such as Hadoop and HBase, are used to reduce the time associated with processing DNA results.

Given how far we have come in such a short time, what should we expect for the future of consumer DNA? The technology is moving so fast that it is almost worthless to predict. But what is clear is that we won’t come out of this genetic revolution the same. We are going to live better, healthier lives, and we are going to learn things about our species and ourselves we never dreamed of. And importantly, putting genetic ethnicity and family connection in the hands of individuals is going to tear down our notion of race and show how we are all family – literally. Maybe we’ll even treat each other a little better.

Ken Chahine is Senior Vice President and General Manager for Ancestry.com DNA.

 

The post DNA and the Masses: The Science and Technology Behind Discovering Who You Really Are appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/dna-and-the-masses-the-science-and-technology-behind-discovering-who-you-really-are/feed/ 0
API Performance Monitorhttp://blogs.ancestry.com/techroots/performance-and-stability-dashboard/ http://blogs.ancestry.com/techroots/performance-and-stability-dashboard/#comments Wed, 15 Jan 2014 07:00:37 +0000 Anders http://blogs.ancestry.com/techroots/?p=1415 My team has been tasked with providing a dashboard for some of our product teams that enables them to aggregate various monitoring systems, logs, metrics, and other forensic tools into one place. While provisioning this dashboard, we discovered we needed a tool that could hit an endpoint, run code against the response, record the results, and… Read more

The post API Performance Monitor appeared first on Tech Roots.

]]>
My team has been tasked with providing a dashboard for some of our product teams that enables them to aggregate various monitoring systems, logs, metrics, and other forensic tools into one place.

While provisioning this dashboard, we discovered we needed a tool that could hit an endpoint, run code against the response, record the results, and do it all on a scheduled basis.

I was assigned to build this tool, which, thanks to modern technologies, only took a couple of days and is now a simple, lightweight node.js server.

One passes in a request object (with the url, method, headers, url parameters, etc.), an interval (e.g. 30000 ms), and a script to run against the response.  The server schedules the task with the interval given, performs the request, compiles and runs the script against the response, and records the results in SQL Server.

We inserted a dozen or so scheduled tasks, and it has been executing flawlessly for the last two weeks.

For example, here is a graph of the response time of one of our servers over the last hour:

Image 2

At some point, a response time of over 800 ms was recorded.  Sometimes a request can just take a long time and there can be a lot of reasons for it – but if it only happened once, it’s usually not worth taking the time to triage.  It is, however, at least worth checking to see if it did only happen once, so here’s a full week:

Image 3

There are spikes of ~150ms, but nothing anywhere close to a full second. It appears to be an isolated incident, so we’ll chalk it up to gremlins and move on.

With this tool available to us, it’s simple to track response times, uptime, downtime, errors, and anything else we deem interesting.

Tools like these save us a lot of time by keeping an eye on things while we’re busy with other priorities, freeing us up to work on adding new features, polishing existing functionality, and removing bugs from the code base.

The post API Performance Monitor appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/performance-and-stability-dashboard/feed/ 1
Why Have a Browser Support Policy?http://blogs.ancestry.com/techroots/browser-support-policy/ http://blogs.ancestry.com/techroots/browser-support-policy/#comments Mon, 06 Jan 2014 21:21:44 +0000 Jeff Lord http://blogs.ancestry.com/techroots/?p=1649 With the growing number of web browsers and mobile devices being used to access content on the internet, it has become increasingly important for organizations to solidify a browser/device support policy. Internally, this type of policy can help with the development and testing of new features and pages by focusing time, effort, and resources on… Read more

The post Why Have a Browser Support Policy? appeared first on Tech Roots.

]]>
With the growing number of web browsers and mobile devices being used to access content on the internet, it has become increasingly important for organizations to solidify a browser/device support policy. Internally, this type of policy can help with the development and testing of new features and pages by focusing time, effort, and resources on a select set of browsers and devices. Externally, users will have a clear understanding of expected functionality and the adjustments they can make to ensure the best experience possible when using the site.

With approximately 2.7 million subscribers and hundreds of thousands of unique visitors a day all using their preferred browsers and devices to access our site, Ancestry needed to define where our teams should focus and prioritize their time. To accomplish this, a committee of development, product, and QA representatives was organized and tasked to develop a browser support policy that accurately reflected the latest industry standards, as well as our particular users’ preferences.

As a result, the following tier system is based not only on the latest global web browser and mobile device usage statistics, but also specific analytics and percentages for our own unique users.

 

browser

Tier 1 – Both Functionality and Visual Design
Browsers accounting for at least 10% or more of unique visitors for two consecutive months will be fully supported. This includes basic functionality as well as proper visual design behavior. These browsers will be tested during regular regression and when pages are changed. All bugs will also be triaged and fixed in the indicated timeframe. Browsers in this category will continue to be fully supported until they account for less than 10% of visitors for two consecutive months. Those browsers will then receive Tier 2 support (until/unless their usage drops below 5% for two consecutive months).

Tier 2 – Functionality
Browsers accounting for 5-10% of unique visitors will receive Tier 2 support. This means visual elements on the site need not appear perfectly, but all features must be functional. Basic testing is required, and major bugs will be triaged and fixed in the indicated timeframe. A browser whose traffic falls below 5% for two consecutive months will receive Tier 3 support.

Tier 3 – No Support
Browsers accounting for less than 5% of unique visitors in two consecutive months will not be individually supported. This group will be separated into two groups: uncommon browsers, and out of date browsers. Since the majority of uncommon browsers tend to follow web standards, they will generally receive an adequate experience on Ancestry.com and therefore shouldn’t be prompted to download a supported browser. Visitors using out of date browsers who can upgrade to a supported browser, however, should be prompted to do so.

As for mobile devices, we have designed a majority of our pages to be responsive to the width of the browser. Pages that have been converted receive Tier 1 support, with all other pages receiving Tier 2 support.

The hope is that with this policy in place, it will save time and effort internally, while providing customers and users with the best experience possible on the browsers and devices they use the most.

The post Why Have a Browser Support Policy? appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/browser-support-policy/feed/ 0
A Fast 11 Years at Ancestry.com: Billions of Historic Records, Millions of Customers and A Boat-Load of Code Laterhttp://blogs.ancestry.com/techroots/a-fast-11-years-at-ancestry-com-billions-of-historic-records-millions-of-customers-and-a-boat-load-of-code-later/ http://blogs.ancestry.com/techroots/a-fast-11-years-at-ancestry-com-billions-of-historic-records-millions-of-customers-and-a-boat-load-of-code-later/#comments Mon, 03 Jun 2013 15:54:33 +0000 Scott Sorensen http://blogs.ancestry.com/techroots/?p=733 I joined Ancestry.com as a developer in June 2002 – exactly 11 years ago – just as Ancestry.com was inventing the family tree technology that differentiates us today. The last 11 years have flown by as our team innovates against the constant challenge of improving our unique and dedicated community’s experience. Looking back over that… Read more

The post A Fast 11 Years at Ancestry.com: Billions of Historic Records, Millions of Customers and A Boat-Load of Code Later appeared first on Tech Roots.

]]>
I joined Ancestry.com as a developer in June 2002 – exactly 11 years ago – just as Ancestry.com was inventing the family tree technology that differentiates us today. The last 11 years have flown by as our team innovates against the constant challenge of improving our unique and dedicated community’s experience. Looking back over that decade-plus-one, I’m amazed at how much has changed in the technology industry, and pleased we’ve been able to adopt and leverage those changes to accelerate our core mission.

When I joined the company in 2002, the first iPhone wasn’t even in development, “android” was a science-fiction term for a form of humanoid robot, Google was working on its first-generation of search tools and Mark Zuckerberg had just graduated from high school. Hadoop was simply the name of Doug Cutting’s son’s stuffed elephant, and clouds were cirrus, nimbus, and cumulonimbus.

Eleven years ago, Ancestry.com had 200,000 subscribers, historical records numbering in the thousands and 40 developers. Today we have 2.7 million subscribers, 11 billion records and images (4 petabytes of data), and our developer corps has grown 10 fold, to about 400. In 2002, our search capability we were using was AltaVista. Now, thanks to our innovative vertical search technology customized for our business, our servers handle 40 million searches every day, and our technology-driven ‘hints’ are generating an impressive 60 percent of user discoveries. Back then, all customers used our software on desktops.

Change in the tech world has fueled change at Ancestry.com. This has always been a technology company, first and foremost. Last year we launched a new direct-to-consumer DNA test, AncestryDNA, that’s caught on fast and is helping subscribers build out and understand their family trees. AncestryDNA is generating exponential amounts of data that we’re managing with Hadoop, MapReduce and Hive.  We’re also exploring narrative technologies to give users another way to turn their family history into compelling stories, and photo recognition software to identify similar facial characteristics of ancestors and thus help users find more family matches. We’re always on the lookout for great engineering talent, and have hired gifted data scientists who are using Natural Language Processing, machine learning predictive analytics to understand future user behavior and enable us to make smarter business decisions. Our teams also innovate in the way we scale and manage our systems to be more robust.

What hasn’t changed since I joined Ancestry.com is our commitment to put the latest technology advances into the hands of the world’s best engineers to improve our customers’ total experience. Eleven years have gone by in a blink, since I’ve been fortunate to experience some amazing challenges and solutions we’ve developed in that time. And I can’t wait to see what interesting problems we’ll be handling in the next 11 years, in the technology industry as a whole and right here at Ancestry.com as we grow our exceptional team to serve the expanding global audience for family history research. It’s been a great ride so far, and it feels like we’re just getting started.

The post A Fast 11 Years at Ancestry.com: Billions of Historic Records, Millions of Customers and A Boat-Load of Code Later appeared first on Tech Roots.

]]>
http://blogs.ancestry.com/techroots/a-fast-11-years-at-ancestry-com-billions-of-historic-records-millions-of-customers-and-a-boat-load-of-code-later/feed/ 1