When you use the MTurk Toolkit, you can Exclude workers by specifying previous studies workers have completed, entering Worker IDs onto a list, or creating a Worker Group. Excluded workers are not eligible to take your study.
When Might I Want to Exclude Workers?
You might want to exclude workers when you’re running multiple studies in the same line of research. Once participants have been exposed to the manipulations or measures in one of your studies, you might want to make sure they do not also participate in similar studies. Regardless of the specific reason you want to Exclude workers, there are several easy ways to do so.
How Do I Setup an Exclude List?
You can Exclude workers by specifying studies workers have previously completed, entering a list of Worker IDs, or creating a Worker Group. All three of these options operate by excluding workers who participated in and were approved for your past studies. If, however, you want to exclude workers who may have seen your previous study but not completed it, there are two options for excluding all workers who were associated with your past study. These options are the Super Exclude feature and creating a Survey Group.
Finally, if you have a group of workers that have provided sub-par data in the past, you can exclude them from all your future studies by placing them onto a Universal Exclude List.
Excluding Workers Who Have Completed Previous Studies
- Exclude Workers from a Previous Study
To Exclude workers who completed a previous study, simply select which study or studies you want to exclude workers from while on the Worker Requirements page during the study setup.
- Exclude Workers Using a List of Worker IDs
To exclude a list of workers using Worker IDs, paste MTurk Worker ID’s (or CloudResearch ID’s) into the Excluded workers box on the Worker Requirements page in the study setup.
TIP! Only 2,000 workers can be Excluded using the Worker ID method. If you need to Exclude more than 2,000 workers, try excluding by study or creating a Worker Group.
- Exclude Workers by Creating a Worker Group
To Exclude workers using a Worker Group, select "Create a new Worker Group" from the Worker Requirements page of the study setup.
On the following page, give your Worker Group a CloudResearch name and choose "Exclude Workers." Then select "Create Worker Group."
Finally, the last page presents options for managing your worker group. From this page select "Add Workers to Group." On the following page copy and paste your list of Worker ID’s into the text box. Then select "Save." You worker group should now appear as an option for exclusion while setting up studies.
Excluding Workers Who Have Merely Seen Previous Studies
When setting up an Exclude list, there are two options that will ensure all workers are completely unfamiliar with your study. They are the Super Exclude feature and the Survey Group. Both of these features exclude any worker who has accepted specified studies rather than workers who accepted, submitted, and were approved for specified studies like the options above.
- Using the Super Exclude Feature
To use the Super Exclude feature, choose which study or studies you would like to exclude previous workers from. Then, check the box next to the Super Exclude feature on the Worker Requirements page during study setup. Now, all workers who accepted one of your previous studies will be excluded from your current study.
- Using a Survey Group
The Survey Groups feature offers a way to exclude workers from multiple studies running simultaneously. Thus, if you or your lab has multiple studies running at the same time and you want to ensure unique workers in each one, you should use the Survey Group feature.
To create a Survey Group, go to the Pro Features page in the study setup. From there, select "Create Survey Group."
Next, you can give your Survey Group a name and add active surveys to the group. Surveys added to the group will be mutually exclusive once the study goes Live. Finally, select the survey group so that it is active when you launch your studies.
NOTE: The Survey Group feature is intended for launching multiple studies at the same time. Studies can only be added to a survey group while they are being created, not once a study is Live or after it has been Completed.