Big Changes!


We’ve worked with a representative group of your peers over the last several months to test and implement some changes to the Keying Tool.  Many of these changes are things you have requested.  Some of these changes are process changes that we feel will allow your contribution to have an even greater impact.
 
We expect the new Keying Tool to be available within the next 24 hours.  So, watch for a download and install the next time you open your keying tool.  The major changes are as follows:
 
Key/Review
 
Historically our process has included a double-blind key with an arbitrator reviewing the differences.  Our assumption has always been that this process provided the best quality.  After extensive testing and review of data we have discovered that indexing quality is most affected by the expertise of the last person who touches the record – whether they are the first and only person or the twelfth person makes no difference.
 
With that in mind, we have decided that going forward most projects keyed in the Ancestry World Archives Project will be indexed in the following manner.  One person will key an image set.  A second, more experienced keyer, will review 100% of the fields keyed per image set.
 
Trusted Keyer
 
This also brings us to the conclusion that a more experienced keyer, who has proven their level of quality, does not need all of their keying reviewed.  So, with these changes, we have also implemented a process we are calling Trusted Keyer.  This means that, after keying a significant volume of records with high quality, most of the records keyed by a Trusted Keyer will pass through the system without additional review.  This will be administered on a per project basis.  Occasional review will still be performed on their work to ensure that they continue to qualify as a Trusted Keyer.
 
By doing this the time and effort you put into keying now has a greater impact on the volume of records being produced.
 
Feedback
 
At this same time we are introducing a much requested feedback feature.  This feature allows reviewers to choose a pre-selected piece of feedback to submit with each image set.  Phase one of this feedback feature allows us to aggregate this information on a per project basis.  Additionally, we will be providing you with personalized quality accuracy per project.  We will begin collecting this information immediately with each image set reviewed but expect to see this feeback available on the Project Pages in early February.
 
If you are an arbitrator, we invite you to try out the new reivew and feedback processes as soon as the new tool has been automatically installed on Thursday.  If you have questions or comments about any of these changes, please leave them here.  We will address them in the coming days.

Until next time – Happy Keying!
 
P.S. – A complete list of all feature changes and bug fixes in the new release of the Keying Tool is coming soon.

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.


Other Posts

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

@Gail (50) Stars show projects that contain Arbitration. No stars are A-review.
Right now there are 3 projects that are a-review

California RR
Pa Nat Originals
Sacramento grand register

That will change as more projects are converted. (not all will be)

@paulmd199, thank you.

I think the loss of the percentage may be regretted by wap.
The percentage made it unique, and was a major factor in why I chose ancestry to give my time keying, now ancestry is no different to other organisations that need keyers.

I liked to know how i was doing, and it was very easy to see if i was doing something wrong.
I am at 95% and at exceptional, so I could go down to 92% or lower before it changes !!
That is too far, think of how many records I would have done wrong to drop 3% + seeing as i key over 400 records a day !!

Please bring back the percentage.

I have been practising reviewing the PA Nat records – and I have to admit that I have been marking as Perfect those records that contain just very small spelling errors, where the writing was very difficult and I could see how a different interpretation might have been made by the original Keyer. If all the reviewers are as lenient as I am, those people who care about their scores will be very happy!

@Tom — thank you for the information — it was very helpful. Most of it confirmed what I am doing already, but I admit I messed up in one area (but not so often). The first name field says not to put in apostrophes (Sam’l), but I did because the surname field says to. So why the exclusion on the first name? Again, thanks for the info.

Experienced indexers can make mistakes!
It is better to have the indexes checked even just once, than to let errors go through. With an accurancy rating of excellent, I goofed and submitted an index before it was finished, and that was today!!

As an arbitrator, I love the feedback part as I have rejected 3 sets recently. I am working on the Buffalo Soldiers and it is difficult to always get it right even for the best keyer. So I like the idea of the “Trusted” keyer to be able to “question” their work. Thanks for listening.

The feedback review will be interesting – I would like to know my mistakes – or at least what type of mistakes I make.
Also a place to comment on a particular page is an excellent idea. I am now an active contributer – hoorah!!

I’m new to this but is there any way to see what mistakes I am making? If I am consistently doing something wrong then a message or email would be helpful to set me straight.

@Wayne (#59). Nowhere just yet. It’s in development. Eventually there will be something on the help pages for the individual projects.

Best advice for now is to read the help files, keying standards, and of course review your work before submitting.

I welcome these changes and will try transcribing again. The old keying system just wasn’t working for me, so I’ll download the new tool and see how it’s improved. Thank you!

I am brain damaged (honest), but I go through life with the attitude “I can do this.” Keying here has become more than tolerable, though. Accordingly, I am an “exceptional” keyer, yet the board moderators for the PA Nats/Decs is claiming that each and every image they are, I guess, arbitrating, are incorrect. Is it Austria; Hungary, Austra; Hungary; or Austria; Hungary; or Austria, or Hungary. None of these are correct, according to the moderators, no matter how hard I try to figure it out. The images I’ve been working on do not contain the names of spouses, nor children, so the images I am keying are now unacceptable. I am not going to provide names in spaces that are not available, like it or not. Most of the images do not have birth dates, arrival dates, and the birth countries, i.e., Poland, are split in ruling powers between Russia, Germany, Austria; Austria-Hungary; Hungary; and Germany. Apparently all the images I have keyed are incorrect, so I’m trying hard to hang in despite the claims by the board moderators that every record received is totally incorrect. This project is not worth the effort any longer. Maybe my brain damage has caught up with me and I’m too stupid to be attempting to work on the World Archives project–yet you claim my submissions are “exceptional”. I can’t ask for help anywhere else, such as contacting Ancestry support/contact, as I cannot “print screen”–all I receive is a request to provide the specific problems I am encountering from the board moderators or arbitrators. I dare not serve as an arbitrator despite my alleged “exceptional” ability because I am clueless as to what the board moderators or arbitrators want. The harder I try, the worse the problems have become. Is this worth it, providing records to the public for free when apparently the “exceptional” images I am keying are all wrong, and not worth considering–so down the line I have no idea what value my work is to the World Archives project any longer. Being brain damaged has it downside, too, as in I cannot afford to pay $29.00 to work on my extensive family genealogy when I am unable to work–although I realize I can key my incorrect images for free. This project has gone well beyond disappointing.

Sue, Not EVERY one. If you have an exceptional rating, your images are fine!

Please consider going back to the per centage bar for accuracy.
Having a “name” rating isn’t a real help for me.

I had not worked on any projects for a while, but recently I started back up. I downloaded the update and it worked for a couple of days, but the last week it has not given me any projects. Giving me error messages. What should I do?

Trusted keyer?? I must agree with the other comments here. It is a dangerous path. As a former legal secretarial supervisor, even my most experienced secretaries made mistakes what with phones and people stopping by to ask questions. When I do my keying, it is usually when no one is home to destroy my train of typing.
Oh, how I wish you would reconsider that portion of the “New Rules”.

Sorry for this second comment. I am a history buff and that has helped me very much in keying some information regarding countries. Austro-Hungarian Empire for instance after World War 1 was broken down into Austria, Poland, Hungary. So Poland did not exist before the end of World War 2. THe same applies to Iraq and Iran. Iraq was part of the Persian Empire (Iran). After World War 2 and the Potsdam Conference, a lot of countries appeared and had their borders re-drawn. What I usually do is look at the town that is listed for certain people and they will tell you if that town was in Russia, Poland or the others mentioned above. I am beginning to drift, so I am signing off.

Oh yes, one more thing. Accuracy has to be the engine that drives this project or it means nothing.
Accuracy, is MOST important.

Paul, I am not going to quote what is on the PA Nats blog verbatim. As I read it, none of the PA keyed images are acceptable, and better performance on the NY Nats. Since my ancestors, with one family exception, arrived and thrived in PA–that was my purpose for working on that project. Additionally, the percentage of work performed on the PA Nats when I began was 23%, and it now is approaching 89%. I’d like to see it completed, but it is nonetheless the most complicated project I’ve ever applied my typing/transcription/keying skills, as I’ve worked on many other projects for other genealogy genres. Peggy, I worked in legal administration for most of the 20 years of my working life–if I weren’t so bent on accuracy, it would’ve stopped long before. I’d like to think I know something about Poland, since half of my ancestry either was born in or arrived from Poland. Poland, sadly, was torn apart by Russia, Germany, and Austria–but it did exist prior, during, and after WWII; it never ceased to exist. If I’m reading too deep between the lines, my apologies. My Polish Jewish ancestry, if any, were not so fortunate. The accomplishment of working on the World Archives project seemed, at one time, worth the effort I poured into it. But now I’m too disgusted with the poorly planned, constantly changing guidelines and feedback, and total confusion. I strongly recommend pulling this entire project together by the seams, by people who are geared towards accomplishment, not splitting hairs to prevent keyers from attempting to participate and likewise feel a sense of achievement.

Sue, The stats say that about 1/4 of images have had no problems.

Most of the rest have had fixable errors.

Arbitrators do fix the errors and submit corrected images. Generally not reject except in extreme cases.

Missing data on the forms is a common one.

Arbitrators occasionally do get frustrated after seeing a long series of images with errors and do occasionally post tactless comments. Again, if your ration is exceptional, you’re doing fine, and have made no major consistent errors.

I agree with the other keyers.Bring back the percentage rating. As keyers we can see how we did on are last and/or next to last project at a glance.This other system does not cut it for us keyers . There is a big gap between Good and Excellent keying.I think as keyers we are all striving to improve and better are keying for The World Archives Project.

@Sue, If I have said anything on the posts that have upset you – I’m truly sorry. That was not my intent.

The sets that I had been arbitrating for the couple of days had not been keyed with much thought. There were senseless error, such as keying July instead of January; keying age and date of birth (a no-no); keying spouse surname when none was listed; not keying date of arrival; not keying date of record.

Then there is a ‘special’ keyer that likes to add Mr to the prefix and the occupation as a suffix.

It was sets like these over and over again for 2 days that were upsetting to me.

Today the sets have been marvelous. Not very many errors and some sets actually were perfect.

I am a creature that strives for perfection. I go by the ‘rules’ that WAP has established.

There are over 7000 sets to be arbitrated. With the new A-review they take longer to arbitrate than the other arbitration process.

Again my apologies.

Karen

Thank you, Karen and Paul. I really do enjoy keying images, despite the intensive eye strain (not all of the time). Your job is far more intense than mine, as arbitrators. This is a complicated project, but at the end of the day I’m glad I’ve found a very interesting activity to keep my brain cells jumping (for better or worse :), and I’m doing it well enough to continue working on it.

Correct me if I am wrong but I think the instruction was key the country exactly as you see it even if that country doesn’t exist anymore.

I also find arbitrating fustrating sometimes. Keyers will key 1580 or 1581 instead of whats in the record 1580/1 or on one I arbitrated recently whitehead instead of whythed but I think back to all the mistakes I made when I was learning what was required and I try to be tolerant.

While I prefer the actual percentage along with the accuracy bar, I can deal with it either way. I think what might be helpful, and would put some keyer’s worries to rest, would be a blog post with the various accuracy ratings trnslated to numbers–like letter grades on your school papers corresponded to certain percentages. I doubt if most keyers know the various levels of ‘accuracy’. In these comments alone, I have seen mention of good, excellent, and exceptional. I am certain there are some below good. If we at least knew the corresponding percent range, i.e. Exceptional=93-100%, and so forth, it would at least clue everyone in on where they fall on the spectrum.

Just read over this very repetitive post! Sorry for carrying on so!

Joni

I do not know how to get an answer to this question ……. when a person is arbitrating and comes to an item that has been labelled “duplicate image” — what are they to do? If it is an image I have personally arbitrated, I agree with the designation. Often times the 2nd page is a duplicate of the first and it is obviously a duplicate. But sometimes, as has just happened, it is not one I have personally seen before. It has been keyed by one keyer and not the other ……. do I just automatically accept that it is a duplicate even though I have no personal knowledge that it is? Please help, this happens far too often. In the past, I have sometimes refused to arbitrate it and sent it back, sometimes I have arbitrated it anyway ….. what is the accepted thing to do?

@joan

Since it’s a rule violation to mark a duplicate outside of the set, you have a couple options.

1) Arbitrate and enter the missing data anyway.
2) reject the keyer

Which one you chose depends on how much is missing and your comfort level.

@paulmd199

Could you direct me to that rule since the answer I received when I asked the question about keying a duplicate image appearing in a subsequent image set was to identify it as a duplicate? I stopped keying the “duplicate” even though it would appear as a duplicate only if the keying of both image sets is being done by the same person. If different people keyed the different image sets, neither would have labeled it as a duplicate.

I’ve noticed that some of the project pages are showing stats in the Project Feedback (an example is Cuyahoga County, Ohio, Marriage Records, 1810-1973). I assume that these are overall stats not my individual stats. Is that correct?

@Karen (79) The Feedback is at present group stats.

The accuracy bar is your own, though.

@Wayne, when the arbitrator sees an image that is marked as a duplicate for which he cannot see the other copy, he will either

A) Reject the keyer’s work and send it back to be redone. or override and key what was omitted. (depending on how much there is)

B) Cancel it with problems. Whereupon the person who reviews the cancellations (I am one) will perform option A.

Option C for arbitrators is kinda bad. Which is to TRUST that the keyer is correct. Which may not be the case and risks losing non-duplicate data. Arbitrators should never pick option C.

Both keyers can be wrong on this. In the Naturalization originals, there is an overlay that blends in well against the image beneath it. Keyers OFTEN the the image with the overlay and mark the image with the overlay lifted as duplicate.

seraphine10, I agree with your comment. I take a lot of time weighing up the spelling options of difficult names and signatures before keying it in, and sometimes I’m still not 100% sure it is accurate. A flag in these instances would be helpful for accuracy.

Hate the new approach. Way too much time (mouse clicks and typing) for the reviewer in fixing simple things (like deleting the addition of periods) when in the past, the odds of one of the keyers getting it done correctly was quite high and it only took 1 click for the arbitrator.

Prediction that reviewer queues will get awfully long. Certainly, more keying vice reviewing for me at least in the future.

Re Duplicate images, I’ve come to appreciate them. having signed some as duplicate in the past, I found by chance in particular with Canadian payrolls, that often the image was dark at the bottom & even obscured some entries completely. all the fiddling with contrast etc. didn’t always make them readable.

Hooray! the faint but readable ‘Duplicate’ often had a clear copy of all that didn’t show on the original image.If it comes up in the same download, I use it to complete my ‘original’ & sign the other as a duplicate.
If it arrives in a different lot, I key it again to include what I couldn’t first time around & hope the Arbitrator doesn’t mind double dipping.

I’m concerned about my work. I’ve been indexing the Alabama Reference Cards and now the Ohio Marriages. One day my accuracy went from exceptional to excellent and no matter how much I read about the projects on wiki, etc., it never improves. Is there a way to find out what I’m doing that I could be doing better? I must not be the only one because the group ratings have gone from 75% to low sixties!

Register of Duties Paid for Apprentices’ Indentures

I’ve been keying and arbitrating these records for a few weeks now and this new method of keying and reviewing is not operational on this project. Wiki says Project Stats are not up and running. As I contributed to beta testing of the new tool I am keen to use it operationally.When may we expect it to become available?