Visualizing Data with Tableau

Posted by Camille Penrod on November 11, 2015 in Big Data, Technical Management

At Ancestry we quickly analyze billions of rows of data to deliver insights from our massive database to internal and external audiences. To do this we need tools that have the right capabilities and can be easily adopted by our tech team. In a recent Q&A video Bill Yetman, VP of Commerce, Data and Analytics… Read more

Data Science Thought Series: How to Embrace Technology Evolution

Posted by Lei Wu on October 27, 2015 in Big Data, Data Science

  Technology evolution is a very common and natural process throughout human history.  Some people like it while some people are hesitant.  Despite attitudes towards technology evolution, new technology will eventually come.  That is how technology trends advance, that is how quality of life advances and that is how human civilization advances. As a technology… Read more

Over one million DNA samples and many new scientific research findings

Posted by Julie Granka on October 2, 2015 in Big Data, DNA, Science

Ancestry science team gives two platform presentations on Friday, October 9th at human genetics conference ASHG With over one million customers who have submitted their DNA, AncestryDNA has the fastest-growing and one of the largest collections of human genetic data around the world.   That amount of DNA data powers the AncestryDNA science team to perform… Read more

Big Data for Developers at Ancestry

Posted by Seng Lin Shee on September 25, 2014 in Agile, API, Big Data, Development, Performance, Technical Management, Testing

Big Data has been all the craze. Business, marketing and project managers like it because they can plot out trends to make decisions. To us developers, Big Data is just a bunch of logs.  In this blog post, I would like to point out that Big Data (or logs with context) can be leveraged by… Read more

The Importance of Context in Resolving Ambiguous Place Data

Posted by Laryn Brown on July 10, 2014 in Big Data, Uncategorized

When interpreting historical documents for the intent of researching your ancestors, you are often presented with less than perfect data. Many of the records that are the backbone of family history research are bureaucratic scraps of paper filled out decades ago in some government building. We should hardly be surprised when the data entered is… Read more

Lessons Learned Building a Messaging Framework

Posted by Xuyen On on July 1, 2014 in Big Data

We have built out an initial logging framework with Kafka 0.7.2, a messaging system developed at LinkedIn. This blog post will go over some of the lessons we’ve learned by building out the framework here at Most of our application servers are Windows-based and we want to capture IIS logs from these servers. However,… Read more

Adventures in Big Data: Commodity Hardware Blues

Posted by Bill Yetman on June 20, 2014 in Big Data

One of the real advantages of a system like Hadoop is that it runs on commodity hardware. This will keep your hardware costs low. But when that hardware fails at an unusually high rate it can really throw a wrench into your plans. This was the case recently when we set up a new cluster… Read more to Present Jermline on DNA Day at the Global Big Data Conference

Posted by Jeremy Pollack on April 9, 2014 in Big Data, Data Science, Development, DNA, Science

Interested in genealogy?  Curious about DNA?  Fascinated by the world of big data?  If so, come check out my talk  at the Global Big Data Conference on DNA day this Friday, April 25 at 4pmPT in the Santa Clara Convention Center!  I’ll cover Jermline, our massively-scalable DNA matching application.  I’ll talk about our business, give a run-through… Read more

Using Mappers to Read and Partition Large amounts of Data from Kafka into Hadoop

Posted by Xuyen On on April 8, 2014 in Big Data

In my previous posts, I outlined how to import data into Hive tables using Hive scripts and dynamic partitioning. However, we’ve found that this only works for small batch sizes and it is not scalable for larger jobs. Instead, we found that it is faster and more efficient to partition the data as they are… Read more

DNA and the Masses: The Science and Technology Behind Discovering Who You Really Are

Posted by Melissa Garrett on March 12, 2014 in Analytics, Big Data, DNA, Science

Originally published on Wired Innovation Insights, 3-12-14. There is a growing interest among mainstream consumers to learn more about who they are and where they came from. The good news is that DNA tests are no longer reserved for large medical research teams or plot lines in CSI. Now, the popularity of direct-to-consumer (DTC) DNA tests… Read more