Short story: Mturk with Django, Amazon registering all HITs as complete whereas Django database shows only half finished.
Long story: I have some Mturk HITs running through a Django application. Turkers take a demographics test, qualifications test, some surveys, and then proceed to the tasks. The tests and surveys are finished once and the turker can complete as many tasks as they like.
I put a few HITs in the Amazon sandbox and everything worked perfectly. Amazon collected the correct information and my Django database collected what it was supposed to collect. So I put up 1000 tasks (100 HITs with 10 tasks per HIT). I expected all of these to be completed with in a day, but after a few days I noticed (when looking in the Django database) that the tasks were not being completed. So I went to the Amazon site to take everything down and saw that ALL of the tasks were finished.
Looking into it further I found that Django stored all the test and survey information for each turker but for some reason only registered a little over 500 tasks as completed, whereas the Amazon site shows that about 975 of them were finished.
What could be happening? My first thought was that most of these tasks were completed around the same time and maybe the Django database (a SQLite database) couldn't handle this much activity? I would think if there were errors in the app then turkers wouldn't be able to access new HITs, but this isn't the case as Amazon is showing they are all complete. As far as I know, no turkers have emailed to complain about errors.
I'm not even sure how to go about debugging this since I'm not seeing any errors when I test the site.
Aucun commentaire:
Enregistrer un commentaire