Big Changes!


We’ve worked with a representative group of your peers over the last several months to test and implement some changes to the Keying Tool.  Many of these changes are things you have requested.  Some of these changes are process changes that we feel will allow your contribution to have an even greater impact.
 
We expect the new Keying Tool to be available within the next 24 hours.  So, watch for a download and install the next time you open your keying tool.  The major changes are as follows:
 
Key/Review
 
Historically our process has included a double-blind key with an arbitrator reviewing the differences.  Our assumption has always been that this process provided the best quality.  After extensive testing and review of data we have discovered that indexing quality is most affected by the expertise of the last person who touches the record – whether they are the first and only person or the twelfth person makes no difference.
 
With that in mind, we have decided that going forward most projects keyed in the Ancestry World Archives Project will be indexed in the following manner.  One person will key an image set.  A second, more experienced keyer, will review 100% of the fields keyed per image set.
 
Trusted Keyer
 
This also brings us to the conclusion that a more experienced keyer, who has proven their level of quality, does not need all of their keying reviewed.  So, with these changes, we have also implemented a process we are calling Trusted Keyer.  This means that, after keying a significant volume of records with high quality, most of the records keyed by a Trusted Keyer will pass through the system without additional review.  This will be administered on a per project basis.  Occasional review will still be performed on their work to ensure that they continue to qualify as a Trusted Keyer.
 
By doing this the time and effort you put into keying now has a greater impact on the volume of records being produced.
 
Feedback
 
At this same time we are introducing a much requested feedback feature.  This feature allows reviewers to choose a pre-selected piece of feedback to submit with each image set.  Phase one of this feedback feature allows us to aggregate this information on a per project basis.  Additionally, we will be providing you with personalized quality accuracy per project.  We will begin collecting this information immediately with each image set reviewed but expect to see this feeback available on the Project Pages in early February.
 
If you are an arbitrator, we invite you to try out the new reivew and feedback processes as soon as the new tool has been automatically installed on Thursday.  If you have questions or comments about any of these changes, please leave them here.  We will address them in the coming days.

Until next time – Happy Keying!
 
P.S. – A complete list of all feature changes and bug fixes in the new release of the Keying Tool is coming soon.

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.


Other Posts

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

The Feedback feature will be most welcome. Looking forward to seeing its addition, and a speedier application of the keying process.

Trusted Keyer– Unreviewed?! THAT didn’t come up in the beta test.

Even expert keyers still occasional make mistakes, omissions and typos. I know I’ve made my fair share.

What exactly is the threshold of accuracy for this?

Even though I feel I have a good accuracy rating, there are times that I hope I’ve interpreted the handwriting correctly. I would suggest that there be a way to indicate that a record should be reviewed by someone else because you don’t feel confident that you are right and that you just took a guess. Perhaps an asterisk or an ‘x’ in a special column to flag that record for review.

Love the new review process. I find it so much easier and faster, and also believe it improves the accuracy of the final product because the arbitrator MUST review each field, and is only concentrating on one keyer’s input at a time.

Looking forward to the Feedback review. It amounts to gaining experience in keying. I note the cynical comment along the way. Anything that would enhance my keying of the World Projects is very welcome at my end.

I’ve got to agree with Georgina, I hope these changes are not just away of keying more records, I thought accuracy was the number one priority? I do think the Feedback Feature is a good Idea should benifit keyer.

Love the arbitrating idea. I agree that allowing the final arbitrator to be someone who is perhaps less skilled is not a great idea. This will hopefully be a great improvement. I would agree with the hesitation by others regarding even trusted keyers to skip arbitration. Even the best may not always be sure on some records.

I entirely agree with paulmd199 about Trusted Keyer. If her makes mistakes I certainly must make more! A dangerous path to follow.

See, I just made one there. I know you’re a ‘he’ Paul!

…and I’m not sure that ‘trusted’ is the correct work. Sounds like a prisoner who keeps to the rules! ‘Experienced’ might be a better phrase.

..there you are again – I meant ‘word’. We all have ‘off’ days!

I agree with Georgine Herd (No. 3). Even though I do my best and my accuracy is fairly high, there are times when I am not 100% sure that I have deciphered a name correctly or selected the appropriate version out of multiple spellings on one document. A flag would just act as an alert for the reviewer.

I, too, believe that though a person is an experience keyer, who reads the instructions, follows the rules, they can still make typo’s. Various handwriting and quality of the an image can play a huge part in a keyers ability to read the image correctly. Hopefully these projects will still have the Ancestry feature of corrections submitted by a researcher as in all the other databases. Earlier databases that Ancestry has on line have some really bad “reads” of handwriting and finding those records has been a huge challenge akin to stumbling around in the dark looking for the door out.

I like the idea of a keyer being able to flag a file that he or she has doubts on, as long as the “trusted” keyer is willing to tell on themselves. In our efforts to “get the info in”, that we want correct info.
I think that the level has to be set fairly high to become a “trusted” keyer. If a person consistently averages high percentages and is willing to flag a doubtful data set, then let’s try this.

Even the keyer that has a great accurary ‘most’ of the time is going to have days that records are hard to read, will miss keying fields, hit the enter button twice and submit an unfinished record, etc.

If I had a 100% accuracy rating (which I don’t) I would still want my input of records to be reviewed by another.

Agree with the notion of a flag, but extend it to a comment field that would identify the keyers concern. These might then reveal a pattern for instruction, or policy.

Hi all,
We have now rolled the new version of the keying tool, v. 96, live.

You can access the Review Instructions here, http://www.ancestry.com/wiki/index.php?title=World_Archives_Project:_What_are_the_steps_to_follow_for_the_Review_process%3F.

Happy keying!

Would it not help people searching for names if with the ones we just cannot honestly decipher, we were able to use a dash or choice of two letters or some such instead of presuming & maybe getting it entirely wrong?
There is no way some of the scrawled names can be deciphered.

I referred to names which may be either of two or three very different names due to the change of one or two letters, which a searcher could pick up on, not totally obscure ones.

Trusted Keyer will pass through the system without additional review.!!!!

O dear :( IMO, NO record should pass through the system unchecked.

By doing this the time and effort you put into keying now has a greater impact on the volume of records being produced.

Yep, more records passing through unchecked = bad practice = bad records.
Even the best keyer has off days.

I too think there should be a flagging process. Even though I get exceptional I still make mistakes sometimes or just can’t work out what that particular word is. Different eyes can sometimes help with terrible handwriting. I often ask my husband what does that say – sometimes he can read it staightaway and then I think of course and other times he can’t either.

I’m agreeing with other comments – NO record should pass into an active database index without review (I’m with Vicki on this).

Agree that there should be some option to ‘flag’ records that are of concern.

Another less worrying comment – although my keying is running at ‘exceptional’ I would still like to see a percentage figure alongside this (if some fluctuation starts in my percentage rate, it suggests that I am doing something wrong or not understanding something which I can then check out)

I agree with Wendy regarding having accuracy percentage as well as “exceptional” or whatever. It`s easy to slip a few accuracy points and never know that you are making mistakes, especially important when keying an unfamiliar project.
And, to echo Paul, I`d like to know the accuracy threshold for being a Trusted Keyer (also agree that it has an unpleasant ring to that title, prefer them to use Experienced Keyer, or Accurate Keyer or something!)

Ito would like to Know what the percentage is to be rated exceptional as I was running at 93% and am now rated exceptional. I dont think 93% is exceptional, It’s good but surley exceptional should be closer to 100%.

I have reviewed several California Railroad Employee records today. Most of them were missing occupations and some of the lines were not keyed. I think we should go back to the Arbitration procedure as the new Review system places too much reliance on the reviewer. Having arbitrated over 100,000 records in the last 90 days I will not be taking nay more part in the Review process.

I have reviesd a few sets of the California Railroad Emplyment Records and they would have been perfect had the keyer read and followed instructions. Yes, there are some projects and instances where the instructions are a bit muddy, but what I am seeing on these are very basic–people consistently neglect to key the occupation! I just finished a ‘full’ set of 2 pages at 28 records per page. No errors except not a single occupation (and it was the same occupation for all!). Honestly, that part of the instructions is not hard to understand–highlighted fields are REQUIRED but enter information in the other fields if it ia available.

I actual like the review process for these records. I found when I arbitrated that a person either had it right for the most part or not. Usually one set waas near perfect and the other wasn’t.

I did read the discription of the WAP changes carefully and it says that MOST projects will be done with single revies. This leaves room for arbitration for projects which are more difficult to interpret. I see no reason why quality would suffer for easily understood and generally easy to read projects that don’t have a zillion fields. OK, those green puddles of ink on a few of the RR records are horrid, but I think if a hundred people looked at them they would see a hundred different things.

I doubt that Ancestry made this change lightly, after all, satisfaction with the end product will determine how many subscribers they retain yearly. I do not know what the ‘industry standard’ is for indexing records. I can say that I use indexas on a variety of sites, and none are anywhere near 100% flawless. That is part of the challenge of genealogy–even the original records have errors. I think we as perfectionists have to sit back and know that there is nothing perfect on this earth.

Please bring back the percentages in the Accuracy area. I like the “Exceptional” remark, but without the percentages, I can’t tell if I’m slipping or not. Thank you.
Kaye

I notice that you have done away with the percentage rating on keyers. Is this to avoid a lot of flack from some keyers that feel they are doing a better job keying than they really are?

I am less concerned about the A review process than the Trusted Keyer one.

Yesterday I had one fellow who was perfect except He did Goerge, every time, for 7 sets in a row.

You can still see your accuracy percentage: just go to the dashboard and click on the project you are working. A box will then show you your percentage.

OOPS! Maybe not…sorry! It was there yesterday, but don’t see it now!

One of the great things we found in the comments of those concerned about the new Trusted Keyer feature is your enthusiasm for and commitment to quality and accuracy. That is undoubtedly why this community does such amazing work!

Don’t sell yourselves short! It has become very clear that all you talented keyers impose nothing but the highest standard on yourselves, which allows us to even attempt these types of innovations.

I want to provide more clarification about Trusted Keyer that will hopefully ease your concerns regarding quality with this feature.

As stated we did not go into this lightly, Ancestry would never sacrifice quality simply to reach a higher number of records so our first and foremost concern centered on quality.

A Trusted Keyer is determined by a combination of keying volume and quality. It is implemented at a project level and one has to qualify per project. Because of the qualification criteria not all projects will have Trusted Keyers and even though you may qualify on one project you may not qualify on others. Although you have been identified as a Trusted Keyer your work is constantly monitored to ensure that you are maintaining this status. Preliminary tests have shown a consistent level of high quality which gave us the confidence to implement this feature across the board.

There have been a number of comments about difficult names to decipher or even the best keyers making mistakes. Sometimes a second set of eyes can come up with a better interpretation of the data. Sometimes ten people could look at a name and come up with eleven different answers or no answer at all. The final answer on an index isn’t dependent upon how many people look at it but upon the decision of the final person who looks at it. If our most experienced keyers can’t interpret the data, having additional reviewers look at it likely won’t help.

In addition the search technology on Ancestry.com is such that all pieces of keyed data are searched and ranked allowing records to surface even not spelled exactly correct on the original record or in the keyed index. We also allow any user (who generally has more context for the accurate spelling of their ancestor’s name) to submit an alternative spelling for any indexed name. Not long ago I experienced this exact scenario. I found a census record of my grandmother and her first name had been indexed as Littie when in fact her name is Lillie. Even though it had been spelled incorrectly I was able to find the record and I submitted a correction so anyone coming after me would know the correct spelling.

Expecting to have 100% accuracy on any index is an unreasonable expectation because the records themselves are not perfect, the record creator made mistakes and the indexer, in spite of their best efforts, will make mistakes. We understand this at Ancestry and are confident that each of you continues to work towards improving your accuracy, keep working at it, read the instructions and know that we have built in features such as the review process and Trusted Keyer to ensure that the final product is at a high level of quality.

One of the recent changes we have made is to aggregate accuracy at a project level (as opposed to just the overall accuracy that we have published to you previously). Beginning in early February, you will be able to view your own accuracy rating on a per project bases on the project pages. This should help you know if your keying on one particular project is adversely affecting your accuracy.

I have said this before, but you indeed are a wonderful group and I am always amazed at your passion for this program.

Thank you and as Crista always says–Happy Keying!

Hi Darla,

Do you happen to have figures on accuracy and volume of what it takes to get this Trusted Keyer status? Is that threshold configurable on a project basis, or is it global? A 95% means different things depending on the project. It’s kinda bad on a typed directory, but actually very good for a badly handwritten census record.

Will we see something on the wiki page that lets us know we do have this status?

Like others, I have sometimes struggled with pages that are washed out and/or scribbled. I complete a higher proportion of ugly pages, for a variety of reasons. Like others, I would like to be able to flag pages I am having a tough time with.

What provisions are in place to deal with the occasional boneheaded stuff, like submitting a page that’s not yet complete, that even good keyers have done? Provided the keyer doesn’t send you an emailed confession of the sin.

I am also curious as to what a “high level of consistent quality” actually means in terms of numbers, and how the conclusion was reached.

Not very sure about all of this, as even good keyers and arbitrators have bad days. The new system appears to be more about volume than accuracy.

I like the old system much better.

At the beginning of any new project there are masses of mistakes made, even by really good keyers. Are allowances going to be made on very new projects.

The unreviewed Trusted Keyer is a concern. I have seen many a time for instance on Perth Valuations where otherwise good keyers are making errors and omitting records particularly towards the end of the record set. These records would therefore be lost forever to researchers on Ancestry.

Also although a keyer may be accurate there is still some confusion on the correct use of cover page and image without data. Would it be possible to remove the image without data option completely and have WAP remove the no data pages during the final stages before going live. This would ensure that no valuable and interesting pages are lost to researchers

I would also like to see the return of the percentage to ensure that progress can be tracked promptly and errors hopefully nipped in the bud should percentages drop

Having read all the comments here carefully including the official ones from ancestry I wonder if it would be a good idea to attach to each project a list of common mistakes that keyers make that the arbitrators have to correct. Some of these I have done in the past because there are no instructions. I have just started as an arbitrator and find many of the mistakes are the same and could be easily corrected before they get to the arbitrator with a little more info to keyers. When I was a new keyer I would have welcomed a list of common mistakes so that I could see where I was going wrong and improve my accuracy easily.

I am also wondering whether arbitrators should be limited to those projects that they have actually keyed. I have put this restriction on myself because I would not feel confident arbitrating any projects I had not actually keyed a great deal.

The most time consuming thing for me as an arbitrator is when the two keyers orders don’t match even though the same records have been keyed. I then have to spend time rearranging both pages before I can begin arbitrating. Please please start with the top of the left page and work down it systematically and then then the right page remembering to start a new section. Some records such as Uk school admissions & Uk postmasters require a new section for each new year starting in January. Check project instructions.

Mostly the same mistakes appear over and over. I can’t emphasise enough key what you see line by line and don’t assume what you dont see actually written, eg don’t change Elizh to Elizabeth even though we all know thats what it should say. Preserve the sanctity of the original record at all times and you will find your accuracy rating improves too. All Genealogists should remember the rule – always check the original record and not just the index.

@s2tree. I have noticed that people start fading towards the end of pages with large volumes to key.

I suspect I may be guilty of that one myself. I try to avoid submitting while sleepy on projects like that.

@darla. One final thought. We are not selling ourselves short. Those of us who are the very best realize that we are not infallible and keep that in mind when working. However confident we may be on our abilities. In fact, those of us who take on the toughest projects have a certain amount of hubris. To be able to say, this will ruin my percentage, but nobody else will key it if I don’t. (is my ego showing? :) )

It means we double check our work ourselves. But a number of otherwise good keyers DO make omissions, and misreads, and occasionally aren’t clear on every rule of every project. Sometimes they get bit by things that occur infrequently and don’t affect percentages enough to notice.

That having been said, there must be a safety net. Someone to filter out the dumb mistakes we all make as humans.

I see constructive comments from all and echo the obvious attention to quality that I read. I share that, shall I say, obsession.

Other than some nervousness about the details, I fully enbrace the direction that the project procedures is taking. We don’t see the statistics but I believe the WAP staff when they say that the point of diminishing returns is almost always reached when one of the better people “touches” the data, whether that be keying or arbitrating.

I have long felt strongly about Darla’s points:
. All the eyes won’t fix the errors that went into the original,
. Sometimes 10 people will have 11 opinions on the correct reading, and
. Search can do near-magical things (agonizing over reading “Gonzales” or “Gonzalas” sometimes doesn’t matter much).

That said, we have to “whup up” on the keyers who do things like only key the required fields. The new feedback mechanism will get us there, probably in the next iteration. In the meantime I have been patient about entering all of those RR Occupations, considering Review closer to being a second keyer than being an arbitrator.

I also agree with the many comments that if I become a trusted keyer on some project I want a way to say “not on this image set” without having to reject the set.

One final comment, risking flames from others – perfection will never be possible and there realistically must be a compromise, forgetting about that last 2% (to make up a number) of “error”, the get the quantity out.

I want to thank WAP for their great work to make this possible and, of course, all of the community for (as I say) doing this instead of Sudoku. (I quit Sudoku when I started WAP.)

Happy WAPping, Tom

I’m confused – my keying tool is still showing an actual percentage for accuracy, although if I get to the dashboard on the actual ancestry site it has the bar and “Accuracy: Exceptional”. Have other keyers not got the percentage showing in the keying tool any more?

No Kate the percentage has gone since the update.
Have you installed the update?

@Kate, it looks like you have not gotten the latest keying tool download. I suggest you try Check for Updates under the Help menu.
Tom

Is the feedback only going to appear on the project pages? Because if the person the feedback is meant for is not reading the project pages they are not going to know what they are doing wrong. Is there a way that you can send the feedback to the individual keyer to let them know what they are doing wrong?

Judy, right now it will not be individualized. After some experience, the hope/plan is to individualize it. The system is moving in the right direction.

This is a great idea! As an arbitrator only the keyed records which don’t match are marked for arbitration. When 2 keyers haven’t read the instructions and make the same mistakes the arbitrator has to check the matching records to ensure they are correct.

I know that there are a lot of responses to this article (me included), but this is a general concern in that it’s hard to find all the keying instructions. Do you look in the genl info? Do you look in the project info? Do you look in Wikipedia? Or do you have to spend time looking at every posting because like this one, there are comments relating to some projects that you may have missed looking somewhere else. (Or is there somewhere just what I need and I haven’t found it yet??)

Thanks for the suggestions, but it is the new version. I don’t really want to lose the percentage anyway, so I’m not bothered about it!

@Georgine, I’m unsure of your level of experience here so sorry if this reply is too elementary.
. the boards are useful but they don’t serve as the best place to learn about keying a project because older posts are not readily visible.
. you probably know but general keying standards are available from the keying tool through Help, View Online Keying Standards
. project-specific keying standards are available, when you have an image set open, with Help, View More About This Project. That is the Wiki people keep talking about.
. there is a general trend to put current information in the Wiki. The Discussion tab is used, on some projects, to record the results of message board discussions. Right now that is up to the initiative of individual members but we should try to get more organized about it. It is little known, but anyone can contribute there.
. Help, View Project Instructions contain links to the Wiki and to

I hope this helps. Tom

I would suggest that all arbitrators check all records whether matched or not. I clicked on a matched record by mistake and discovered that although the two records matched they were still entered in the wrong places. Since then have discovered a lot of records which match each other but not the original image. A bit alarming! I now check them all, a pain I know.

What are all the red * asterisk in the arbitration now Arbitration and Review image sets.

Just curious as only 1 set on my list does not have the red *

@Gail (50) Stars show projects that contain Arbitration. No stars are A-review.
Right now there are 3 projects that are a-review

California RR
Pa Nat Originals
Sacramento grand register

That will change as more projects are converted. (not all will be)

@paulmd199, thank you.

I think the loss of the percentage may be regretted by wap.
The percentage made it unique, and was a major factor in why I chose ancestry to give my time keying, now ancestry is no different to other organisations that need keyers.

I liked to know how i was doing, and it was very easy to see if i was doing something wrong.
I am at 95% and at exceptional, so I could go down to 92% or lower before it changes !!
That is too far, think of how many records I would have done wrong to drop 3% + seeing as i key over 400 records a day !!

Please bring back the percentage.

I have been practising reviewing the PA Nat records – and I have to admit that I have been marking as Perfect those records that contain just very small spelling errors, where the writing was very difficult and I could see how a different interpretation might have been made by the original Keyer. If all the reviewers are as lenient as I am, those people who care about their scores will be very happy!

@Tom — thank you for the information — it was very helpful. Most of it confirmed what I am doing already, but I admit I messed up in one area (but not so often). The first name field says not to put in apostrophes (Sam’l), but I did because the surname field says to. So why the exclusion on the first name? Again, thanks for the info.

Experienced indexers can make mistakes!
It is better to have the indexes checked even just once, than to let errors go through. With an accurancy rating of excellent, I goofed and submitted an index before it was finished, and that was today!!

As an arbitrator, I love the feedback part as I have rejected 3 sets recently. I am working on the Buffalo Soldiers and it is difficult to always get it right even for the best keyer. So I like the idea of the “Trusted” keyer to be able to “question” their work. Thanks for listening.

The feedback review will be interesting – I would like to know my mistakes – or at least what type of mistakes I make.
Also a place to comment on a particular page is an excellent idea. I am now an active contributer – hoorah!!

I’m new to this but is there any way to see what mistakes I am making? If I am consistently doing something wrong then a message or email would be helpful to set me straight.

@Wayne (#59). Nowhere just yet. It’s in development. Eventually there will be something on the help pages for the individual projects.

Best advice for now is to read the help files, keying standards, and of course review your work before submitting.

I welcome these changes and will try transcribing again. The old keying system just wasn’t working for me, so I’ll download the new tool and see how it’s improved. Thank you!

I am brain damaged (honest), but I go through life with the attitude “I can do this.” Keying here has become more than tolerable, though. Accordingly, I am an “exceptional” keyer, yet the board moderators for the PA Nats/Decs is claiming that each and every image they are, I guess, arbitrating, are incorrect. Is it Austria; Hungary, Austra; Hungary; or Austria; Hungary; or Austria, or Hungary. None of these are correct, according to the moderators, no matter how hard I try to figure it out. The images I’ve been working on do not contain the names of spouses, nor children, so the images I am keying are now unacceptable. I am not going to provide names in spaces that are not available, like it or not. Most of the images do not have birth dates, arrival dates, and the birth countries, i.e., Poland, are split in ruling powers between Russia, Germany, Austria; Austria-Hungary; Hungary; and Germany. Apparently all the images I have keyed are incorrect, so I’m trying hard to hang in despite the claims by the board moderators that every record received is totally incorrect. This project is not worth the effort any longer. Maybe my brain damage has caught up with me and I’m too stupid to be attempting to work on the World Archives project–yet you claim my submissions are “exceptional”. I can’t ask for help anywhere else, such as contacting Ancestry support/contact, as I cannot “print screen”–all I receive is a request to provide the specific problems I am encountering from the board moderators or arbitrators. I dare not serve as an arbitrator despite my alleged “exceptional” ability because I am clueless as to what the board moderators or arbitrators want. The harder I try, the worse the problems have become. Is this worth it, providing records to the public for free when apparently the “exceptional” images I am keying are all wrong, and not worth considering–so down the line I have no idea what value my work is to the World Archives project any longer. Being brain damaged has it downside, too, as in I cannot afford to pay $29.00 to work on my extensive family genealogy when I am unable to work–although I realize I can key my incorrect images for free. This project has gone well beyond disappointing.

Sue, Not EVERY one. If you have an exceptional rating, your images are fine!

Please consider going back to the per centage bar for accuracy.
Having a “name” rating isn’t a real help for me.

I had not worked on any projects for a while, but recently I started back up. I downloaded the update and it worked for a couple of days, but the last week it has not given me any projects. Giving me error messages. What should I do?

Trusted keyer?? I must agree with the other comments here. It is a dangerous path. As a former legal secretarial supervisor, even my most experienced secretaries made mistakes what with phones and people stopping by to ask questions. When I do my keying, it is usually when no one is home to destroy my train of typing.
Oh, how I wish you would reconsider that portion of the “New Rules”.

Sorry for this second comment. I am a history buff and that has helped me very much in keying some information regarding countries. Austro-Hungarian Empire for instance after World War 1 was broken down into Austria, Poland, Hungary. So Poland did not exist before the end of World War 2. THe same applies to Iraq and Iran. Iraq was part of the Persian Empire (Iran). After World War 2 and the Potsdam Conference, a lot of countries appeared and had their borders re-drawn. What I usually do is look at the town that is listed for certain people and they will tell you if that town was in Russia, Poland or the others mentioned above. I am beginning to drift, so I am signing off.

Oh yes, one more thing. Accuracy has to be the engine that drives this project or it means nothing.
Accuracy, is MOST important.

Paul, I am not going to quote what is on the PA Nats blog verbatim. As I read it, none of the PA keyed images are acceptable, and better performance on the NY Nats. Since my ancestors, with one family exception, arrived and thrived in PA–that was my purpose for working on that project. Additionally, the percentage of work performed on the PA Nats when I began was 23%, and it now is approaching 89%. I’d like to see it completed, but it is nonetheless the most complicated project I’ve ever applied my typing/transcription/keying skills, as I’ve worked on many other projects for other genealogy genres. Peggy, I worked in legal administration for most of the 20 years of my working life–if I weren’t so bent on accuracy, it would’ve stopped long before. I’d like to think I know something about Poland, since half of my ancestry either was born in or arrived from Poland. Poland, sadly, was torn apart by Russia, Germany, and Austria–but it did exist prior, during, and after WWII; it never ceased to exist. If I’m reading too deep between the lines, my apologies. My Polish Jewish ancestry, if any, were not so fortunate. The accomplishment of working on the World Archives project seemed, at one time, worth the effort I poured into it. But now I’m too disgusted with the poorly planned, constantly changing guidelines and feedback, and total confusion. I strongly recommend pulling this entire project together by the seams, by people who are geared towards accomplishment, not splitting hairs to prevent keyers from attempting to participate and likewise feel a sense of achievement.

Sue, The stats say that about 1/4 of images have had no problems.

Most of the rest have had fixable errors.

Arbitrators do fix the errors and submit corrected images. Generally not reject except in extreme cases.

Missing data on the forms is a common one.

Arbitrators occasionally do get frustrated after seeing a long series of images with errors and do occasionally post tactless comments. Again, if your ration is exceptional, you’re doing fine, and have made no major consistent errors.

I agree with the other keyers.Bring back the percentage rating. As keyers we can see how we did on are last and/or next to last project at a glance.This other system does not cut it for us keyers . There is a big gap between Good and Excellent keying.I think as keyers we are all striving to improve and better are keying for The World Archives Project.

@Sue, If I have said anything on the posts that have upset you – I’m truly sorry. That was not my intent.

The sets that I had been arbitrating for the couple of days had not been keyed with much thought. There were senseless error, such as keying July instead of January; keying age and date of birth (a no-no); keying spouse surname when none was listed; not keying date of arrival; not keying date of record.

Then there is a ‘special’ keyer that likes to add Mr to the prefix and the occupation as a suffix.

It was sets like these over and over again for 2 days that were upsetting to me.

Today the sets have been marvelous. Not very many errors and some sets actually were perfect.

I am a creature that strives for perfection. I go by the ‘rules’ that WAP has established.

There are over 7000 sets to be arbitrated. With the new A-review they take longer to arbitrate than the other arbitration process.

Again my apologies.

Karen

Thank you, Karen and Paul. I really do enjoy keying images, despite the intensive eye strain (not all of the time). Your job is far more intense than mine, as arbitrators. This is a complicated project, but at the end of the day I’m glad I’ve found a very interesting activity to keep my brain cells jumping (for better or worse :) , and I’m doing it well enough to continue working on it.

Correct me if I am wrong but I think the instruction was key the country exactly as you see it even if that country doesn’t exist anymore.

I also find arbitrating fustrating sometimes. Keyers will key 1580 or 1581 instead of whats in the record 1580/1 or on one I arbitrated recently whitehead instead of whythed but I think back to all the mistakes I made when I was learning what was required and I try to be tolerant.

While I prefer the actual percentage along with the accuracy bar, I can deal with it either way. I think what might be helpful, and would put some keyer’s worries to rest, would be a blog post with the various accuracy ratings trnslated to numbers–like letter grades on your school papers corresponded to certain percentages. I doubt if most keyers know the various levels of ‘accuracy’. In these comments alone, I have seen mention of good, excellent, and exceptional. I am certain there are some below good. If we at least knew the corresponding percent range, i.e. Exceptional=93-100%, and so forth, it would at least clue everyone in on where they fall on the spectrum.

Just read over this very repetitive post! Sorry for carrying on so!

Joni

I do not know how to get an answer to this question ……. when a person is arbitrating and comes to an item that has been labelled “duplicate image” — what are they to do? If it is an image I have personally arbitrated, I agree with the designation. Often times the 2nd page is a duplicate of the first and it is obviously a duplicate. But sometimes, as has just happened, it is not one I have personally seen before. It has been keyed by one keyer and not the other ……. do I just automatically accept that it is a duplicate even though I have no personal knowledge that it is? Please help, this happens far too often. In the past, I have sometimes refused to arbitrate it and sent it back, sometimes I have arbitrated it anyway ….. what is the accepted thing to do?

@joan

Since it’s a rule violation to mark a duplicate outside of the set, you have a couple options.

1) Arbitrate and enter the missing data anyway.
2) reject the keyer

Which one you chose depends on how much is missing and your comfort level.

@paulmd199

Could you direct me to that rule since the answer I received when I asked the question about keying a duplicate image appearing in a subsequent image set was to identify it as a duplicate? I stopped keying the “duplicate” even though it would appear as a duplicate only if the keying of both image sets is being done by the same person. If different people keyed the different image sets, neither would have labeled it as a duplicate.

I’ve noticed that some of the project pages are showing stats in the Project Feedback (an example is Cuyahoga County, Ohio, Marriage Records, 1810-1973). I assume that these are overall stats not my individual stats. Is that correct?

@Karen (79) The Feedback is at present group stats.

The accuracy bar is your own, though.

@Wayne, when the arbitrator sees an image that is marked as a duplicate for which he cannot see the other copy, he will either

A) Reject the keyer’s work and send it back to be redone. or override and key what was omitted. (depending on how much there is)

B) Cancel it with problems. Whereupon the person who reviews the cancellations (I am one) will perform option A.

Option C for arbitrators is kinda bad. Which is to TRUST that the keyer is correct. Which may not be the case and risks losing non-duplicate data. Arbitrators should never pick option C.

Both keyers can be wrong on this. In the Naturalization originals, there is an overlay that blends in well against the image beneath it. Keyers OFTEN the the image with the overlay and mark the image with the overlay lifted as duplicate.

seraphine10, I agree with your comment. I take a lot of time weighing up the spelling options of difficult names and signatures before keying it in, and sometimes I’m still not 100% sure it is accurate. A flag in these instances would be helpful for accuracy.

Hate the new approach. Way too much time (mouse clicks and typing) for the reviewer in fixing simple things (like deleting the addition of periods) when in the past, the odds of one of the keyers getting it done correctly was quite high and it only took 1 click for the arbitrator.

Prediction that reviewer queues will get awfully long. Certainly, more keying vice reviewing for me at least in the future.

Re Duplicate images, I’ve come to appreciate them. having signed some as duplicate in the past, I found by chance in particular with Canadian payrolls, that often the image was dark at the bottom & even obscured some entries completely. all the fiddling with contrast etc. didn’t always make them readable.

Hooray! the faint but readable ‘Duplicate’ often had a clear copy of all that didn’t show on the original image.If it comes up in the same download, I use it to complete my ‘original’ & sign the other as a duplicate.
If it arrives in a different lot, I key it again to include what I couldn’t first time around & hope the Arbitrator doesn’t mind double dipping.

I’m concerned about my work. I’ve been indexing the Alabama Reference Cards and now the Ohio Marriages. One day my accuracy went from exceptional to excellent and no matter how much I read about the projects on wiki, etc., it never improves. Is there a way to find out what I’m doing that I could be doing better? I must not be the only one because the group ratings have gone from 75% to low sixties!

Register of Duties Paid for Apprentices’ Indentures

I’ve been keying and arbitrating these records for a few weeks now and this new method of keying and reviewing is not operational on this project. Wiki says Project Stats are not up and running. As I contributed to beta testing of the new tool I am keen to use it operationally.When may we expect it to become available?