The Basics About Arbitration and Review

We’ve had several thousand people join us over the last few months.  We are so excited that you have become part of our community of keying contributors.  You are doing great work!

This influx of new keyers has created a unique challenge for us, however. We are getting quite the backlog in arbitration and review. Each time I mention that I get a few questions about what that means. Today’s post answers all those questions – or maybe, just some of those questions, with an opportunity to get all of your questions answered.

Quality Assurance
In order to ensure quality control we have in place a system whereby experienced keyers review or arbitrate the work of newer keyers. In addition to this reviewers also update the discussion tab on the project pages in the wiki with information about the common errors they see being made by keyers in order to help us all improve our keying.

Some projects are double-blind keyed and then arbitrated. This means two contributors key the same image set without seeing the work of the other. Once submitted, an automatic process determines which fields match and which fields don’t. An arbitrator then looks at the fields that don’t match, determines which keyer is correct or keys in the correct answer themselves.

Some projects are only keyed once and then reviewed. This means that after the keyer has submitted a completed image set the reviewer is required to check every single field to ensure that it was keyed accurately.

Current eligibility requirements to become an arbitrator/reviewer are that you have keyed a minimum of 2000 records in the past 90 days that have been arbitrated/reviewed and have maintained an accuracy rating of exceptional.

That’s the basics of how it works. If you think you might be eligible to be an arbitrator or a reviewer and want to learn more, we are offering a couple of informational sessions this week to answer all your questions. Both sessions will be held this Wednesday (June 29th). The first one will be at 11:00 am Mountain Time (5:00 pm GMT). The second one will be held at 6:00 pm Mountain Time (12:00 am GMT). Each session will last about 45-50 min and you will be able to ask whatever questions you have about how arbitration and review work.

If you are interested in joining us for one of those sessions, please send an email to us at and be sure to indicate which time works best for you. We will then send you a link to the meeting.

Until next time – Happy Keying!

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.

Other Posts

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

While the arbitration process doesn’t make you check every line. I’ve found its better if you do. And actually easier for long unsorted lists of names. It’s too easy to lose your place when the tool simply skips matching entries.

Also the automatic matching isn’t quite correct, the wrong entries are often chosen when the difference is in punctuation, capitalization and spacing.

To set this up go to tools – options – and check the box that says “review all fields in arbitration”.

Would love to help here, have really just now acknowledged the diffirend and have been changin. Would love to help and enjoy it, but a little money would help by now.

Whatever happened to notification of the link to the “information session”? I responded by email, as instructed, that I wished to participate. It is now 8:44 pm on June 29. I have yet to receive the link to the discussion. I’m very disappointed.

Hope you can offer this again with a little more weekend and all. As a new reviewer I would love to have a forum to ask a few questions.

As a new keyer, I am wondering if there is a place to get feedback on the errors they are finding in my work. I went from an “exceptionl” rating for the first few days to “great job” the next how do I find out what my errors were and if it was a consistant error that I would just continue to repeat. I’m a perfectionist and hate making errors. If anyone knows where to find this info please let me know.

Good info – thanks – however what happens when the arbitration on the project you are entering is behind to an extent that you don’t have a current rating? Also 2000 entries involves less time commitment on some projects than others – some have pages to read that have no entries to be made etc etc