Feedback Follow-Up


A couple of weeks ago we discussed new keyers and how best to provide them with encouragement. You had some fantastic feedback! Today I’d like to address a few of those items.

One Help Resource To Rule Them All
Several of you have expressed frustration that you have to hunt for help. We would like to remind you that the Project Pages on the wiki are THE one-stop resource for any project. Read the project page. Check the discussion tab. If you still can’t find the answer you are looking for, add a question to the appropriate section on the discussion tab. Be sure to click WATCH on the top right on the project pages you are interested in so you will be notified of any updates to that page. No more hunting through endless message board threads.

 

Personalized Feedback
We have received hundreds of requests for personal feedback and at least as many suggestions about how to implement it or not implement it.

This is not so easy as you might suspect, from a product standpoint or from a process standpoint. We have listened to your suggestions. We have put some of our best minds to work on a solution. We are very mindful of the fact that we won’t please everyone no matter what we do. BUT, we are working towards a solution that we think will meet your needs. Stay tuned!

 

Project Level Feedback
Meanwhile, there are many of you still not aware that you even have project level feedback available. If you go to the wiki and then click on any project you have keyed on, you will find two pertinent things.

On the right-hand side of the page you will see three widgets. The middle widget is called, Project Feedback. When our experienced keyers finish reviewing an image set, they select one of these options from a list as a way to classify the quality of the image set. Here in Project Feedback you will find a summary of the top quality issues our reviewers are finding. For more details about the common errors reviewers are finding, click on the discussion tab. Most discussion pages will have a “Common Keying Errors Found By Reviewers” section. If you are concerned about your accuracy rating, this is a great place to go to see what you might be doing inaccurately.

The bottom widget on the right is called Project Stats. This shows how many records you have keyed for this specific project. It also shows your accuracy rating for this specific project. If there is no accuracy rating showing it is likely that the records you keyed have not yet been reviewed. This does not negatively affect your overall rating. All image sets are a NULL value (not a 0% value) until they have been arbitrated.

Please note: We are aware that there are a very few projects where this is not working properly. We should have this fixed within the next few days.

 

Better Examples, Training or Tutorials
Several of you suggested that you would like to SEE how something should be keyed not just be TOLD how to key it. We have spent some time over the past couple of weeks testing video solutions that we can embed on the project pages in the wiki. These will be brief videos of a member of the AWAP staff talking their way through HOW they are keying an image set.

Our hope is that this will give new keyers greater confidence as they join us. We also hope that as more experienced keyers start working on new projects this will help us all interpret the instructions uniformly from the beginning.

We’ll make an announcement here on the blog once we have some of those videos up and running.

 

What else do you think might help new keyers? Keep those suggestions coming.

Until next time – Happy Keying!

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.


Other Posts

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

Most errors from new keyers are not typos! They are omissions, wrong form types, not following basic instructions per the drop downs. Every project has excptions to the rules, but the “key what you see” is usually right (which isn’t always easy, I know).
The almost always exceptions to “key what you see” are months and gender which are spelled out. Punctuation is omitted except per project instructions…key on..practice makes perfect!

I haven’t yet found a project where the project level feedback is working. Mine always list the review errors but they all say 0.0% and project stats says its loading but never actually loads anything. Similarly the discussion pages always says “There is currently no text on this page”. As for the watch button I haven’t found one.

Scrap that last bit just found the watch button.

Paul MD suggested that it was unlikely that Ancestry would release any more UK records until the Perth records were more advanced. I think this would be a big mistake. UK keyers are unlikely to key projects that they don’t feel very involved in – not many people have links to Scotland. It is clearly not a very popular project. I have keyed quite a few of the tax and rent records as I could see it was slow and I do have links to Scotland but I can see why its not popular as its very boring – the same names appearing over and over. I think it would be a mistake to try and force a project ahead by withhodling other records. There are far too many other organisations requiring keyers.

Although I agree with Lyn that it would be nice to have some more English(and Welsh) records to key, I just love keying and I would also like to contribute to other people being able to find their ancestors. My family tree is almost finished and a great portion of the records I found were on Ancestry.This would have been very difficult without the people who keyed them in the first place,especially as many of my family moved the United States.

As for helping other keyers,the video is a great idea,but I think the big problem is getting keyers to read the Wiki. We all learn by our mistakes and the personalized feedback will be very welcome whatever form it takes.

I find the “Key what you see”rule is most often ignored in the London School Admittance Project.
Some keyers insist on assuming that the person listed as parent/guardian has the same surname as the child and that in the year column 87(etc)should be keyed as 1887

All the improvements are nice, but the majority of errors I see are obviously due to keyers not reading the instructions. Not typos, not difficulty reading handwriting, etc., but simply not following the instructions.

I agree with Lyn. I have cut down on the amount of records I key/arbitrate as I’m not interested in the currrent projects.

First, I’d like to apologize in advance if my tone offends. I’m recovering from a bout of food poisoning. And am a little bit cranky. Blame whatever was in that artichoke dip.

Put yourself in the position of a project manager who gets to decide what wap keys You find yourself with a few projects that aren’t as popular as you’d hoped. In fairness, you had a reasonable expectation that the Scotland stuff WOULD be popular. Gretna Green (records of elopements) was done very quickly. Despite some image quality problems.

Your decision was to let several projects that competed for resources run its course. And cancel one further, Get it professionally keyed.

When projects are cancelled. You make people mad. Some may leave wap entirely. And question whether wap itself is viable. After enough cancellations, executives start doubting too.

Cancelling the Scotland stuff would have been an even more dangerous decision. I might have considered leaving. Hoping this doesn’t sound conceited, but the services I provide to the wap community are extensive, and go I think would weigh heavily on decisions to cancel projects where I’ve made a substantial commitment. While I’m confident wap would have survived my loss, it may well have been a factor. I am not the only other person involved with Scotland whose potential loss would weigh on that decision.

Scotland has begun to move at a much faster pace now that there are fewer competing projects. There is now real hope of finishing it in the near term

Next on the queue are a number of holocaust projects. Which promise to be fascinating.

After that, given the Interest in UK, id say its a fair bet you will see more. Possible Ireland, since we haven’t done an Irish project yet.

You’ll all be getting desert after you finish your vegetables.

The usual caveat is that I am not WAP, I don’t speak for them. Not even the project managers are 100% sure of the course that the project will take over the next few months.

@ Paulmd199, we would all miss your wisdom and input, so stay right here please.

The biggest things I have found is not keying as seen, not reading the instructions even on the keying tool, the wiki or just reading the boards available for each project country.

I would suggest that new keyers are given instructions where to find the wiki and the boards as soon as they sign up for keying. Also encourage new keyers to not be afraid to ask questions on the relevant projects. Maybe in a welcoming email to begin with.

Most of us actually enjoy helping new keyers and arbitrators. I would have been lost without this help when I started keying.

My rating has been consistently Exceptional, but I’ve been watching it after keying some ridiculous mostly illegible Erie Co NY tax forms. Yup. Today it dropped to Excellent. I know intellectually that the rating doesn’t matter, but it does seem a slap to punish the hours – literally – I spent trying to decipher those pages. What I wish for would be a chance to explain my decisions. I understand the “key what you see”. But I also understand I help no one if I mark every field illegible, or mark it wrong just because the wrong answer is more obvious. So I use all the adjustments available -lighter, darker, larger, etc. And then I search ancestry.com for other records of the same person that might give me clues. I do not use the info from the other records as my answer, but I do look at it for comparison, and have many “ah ha!” moments, when I realize which letter I’m reading wrong, or that what looks like “Charles” at first take is really “Christof”. When I’m forced to make an arbitrary decision, I try to go in the direction of being accurate. But many times I wish there were a box where I could justify my decision, because I am pretty sure that the arbitrator isn’t going to see it, especially if they are working quickly to keep their numbers high. It does quickly become discouraging.

One other idea: It would be nice if the arbitrator had an option when selecting “keyer A” or “keyer B”, to click a “do not penalize” box when the decision is unclear or arbitary, so that we aren’t penalized for genuine effort, but only for carelessness.

@Linda My rating has been at excellent for a while too (Its actually 91, one point below the exceptional threshold), an account of various illegible images.

I could just key a couple of city directories and jack it up, if it bugged me. (they need love too), But I prefer to work the harder projects, so it comes with the territory.

I know what Erie County census looks like, so if you’re managing to beat through them anyway, I say you’re doing just fine.

As a newbie myself, I would just like to caution everyone on the assumption that inaccuracies with newbies are due to them not reading the directions. I am a HIGH-level of attention to detail person, and I have thoroughly read, and re-read, the keying standards, the individual project instructions, the wiki pages, and the form field instructions that pop-up for each individual field. However, even with that I still cannot seem to improve my rating and that really frustrates someone like me. I keep running into “one-off” scenarios where I either have to ask for assistance from the forum or make a judgment call since there is no help available on the wiki page for my particular problem. It would be helpful to have a Q&A section on the wiki page rather than trying to hunt through the forums to see if someone has already asked the question.

I believe I am getting the actual data correct, but I must be inputting it wrong? I have no way of knowing since I can’t tell what I’m actually doing incorrectly.

Sometimes the learning curve is working with the software over time and getting used to the functionality. Other times, it’s getting actual feedback on what to improve when you “need improvement”. My goal is to get to exceptional, but at this point I’m a little discouraged.

P.S. I have also run across conflicting instructions on the wiki page vs. the form field instructions in the software. This is enough to drive someone to drink (but not while keying). :)

Having just signed up this week, I have found this thread very interesting. As a newbie I am hesitant to key too many records until I get some feedback on quality (I have only submitted a few hundred and am now waiting on the feedback before I move on as they could be incorrect).
If other new keyers have the same concerns, perhaps it may be useful to target new keyers for quick turnaround on feedback?
@PaulMD – agree with your comments on UK records (I signed up hoping to help with Irish records as they are as rare as hens teeth but happy enough to work away on US records in the interim – all record databases are valuable, you never know when you’ll need one!)

@Mags

Depending on what projects you picked, it could be a long wait. There are are backlogs in review for a number of projects.

Best advice is to keep keying, ask for help on the wiki discussion page, the message boards, or email support (worldarchivessupport@ancestry.com) in the interim. Individual feedback isn’t up and running yet (you do get an accuracy rating). It’s in development.

As long as your records are substantially complete, reviewers will fix your boo-boos.

@Lyn Offord
“project stats says its loading but never actually loads anything”

I thought the same thing when I visited that page without logging in, but when I logged in as a member and then visited that page, the loading proceeded OK.
HTH, CNS

I get very frustrated when I see my accuracy falling. I am trying to do the correct thing but clearly it is wrong. I would be keen if I could see my work and the verfied data (just as the second keyer sees it) so I can see what I am doing wrong.

@paulmd
Many thanks for your advice – I will forge ahead accordingly (in the hope that there are not too many boo-boos for others to fix)

Where can I find the wiki pages? Is that connected to the downloads, to the help tab, or to the desktop or homepage? I’ve looked, and when I need it most is when I am keying a download, and want to ask a question and then continue. Is that possible?

I know the keying/arbitrating is kept secret, but would it be possible for the 2nd keyer or the arbitrator to sent a message to the 1st or the 2nd keyer about consistent errors? There are some things that keep happening over again, that the keyers need to know to improve their keying and their ratings.

@karen
a direct link is here

http://www.ancestry.com/wiki/index.php?title=Ancestry_World_Archives_Project

you can select the article for whichever project your or working on, or one of the more generic articles such as the keying standards, etc

when keying, you can also select
help->view more about this project
to be taken to the help article for the project in question.

An individual feedback system is being developed.

@paulmd199, how do you work out the percentage on your stats, you stated yours is excellent at 91%.

I know they have different settings now, where we used to have the percentage. Just curious as to how to work it out. Mine is at exceptional, only because I have been working on the one project for a fair while now. I know it will change and fluctuate when I start a new project, but would still like to know my percentage rate.

The overall number is buried in the config files. :)

I’ve not found a method of obtaining the individual project percentages.

Message me offline if you really want to know.

@collier smith
Thanks for that problem solved.

My project stats still say they are loading and I’m not sure who I should ask for help. I log off every night and log back in every morning so I’m not quite sure if it’s just my page or maybe the project. I have been working on Pennsylvania U.S. Direct Tax Lists. Any advice?

Crista -
After reading this post I was able to see the 3 widgets for the project I am working on (NY Naturalizations), but for the past few days I have been unable to get the 2nd 2 widgets to load. Is it just me or is there a problem. I have tried in both Firefox and IE with the same effect. The 2nd widget called Project Stats just sits spinning and doesn’t load. Any ideas?
Thanks.

Old lady doesn’t have website

I am having a hard time with images that are either very hard to read or cannot be read. I have a feeling it is effecting my score. Although I am not putting much stock in the scoring-other people seem to. Anyway, I cannot get the images to come up enough for me to read them. I have fiddled with the quality of the image, but to no avail. So my question is this: if I select a image and it is something that I just cannot read-does this count agaist me?

I to am a new keyer, and as I read this my big response is “What in the world is Wiki” and why isn’t there a quick place beginners can go for advice in how to key something. Quick in, ask the question, get answer, quick out and go on with correct keying. For instance I was keying the California Rail Road employee records. Top of the page says they are all dining car employees, first column has the name in it the next column has Occupation, what I am finding is that only the top name has an occupation listed. Waiter. ther e are no ditto mars in the column below that first entry. So if I am reading the instructions correctly of “key what you see” I only list an occupation for the first employee, from then on it is just a list of names. There isn’t even a place to show that they are all dining car employees when you enter the dates etc. from the header. Anyone looking at the list or pulling up an ancestors record would only know that they worked for the railroad, but not anything about what they did there or where they worked. All I need is a place to go ask the quick question should I repeat the occupation of waiter for all. get an answer then get back to work. Also as a new keyer the first few days I got “exceptional” rating, the next day I was down to “good work”. I try very hard to be accurate, going back over everything twice, being slow and methodical. double checking everything. so If my rating has gone down, then I must be doing something very wrong, but have no way to know what and will only continue to repeat the mistake. Am I filling out the wrong form, missing a piece of info somewhere on the form ? There should be some feedback to newbies to help them improve. Once they are on their way correctly, then let them go on alone.

As I read through the reader comments I noted that there was a concern brought up in 2011. I am finding that I have the same issue as back then. The ‘Project Stats’ states that it is loading, but it never loads. It just shows the small green wreath circling. Is this only happening to a few individuals or a general issue?