Excluding Workers

When you use the MTurk Toolkit, you can Exclude workers by specifying previous studies workers have completed, entering Worker IDs onto a list, or creating a Worker Group. Excluded workers are not eligible to take your study.

When Might I Want to Exclude Workers?

You might want to exclude workers when you’re running multiple studies in the same line of research. Once participants have been exposed to the manipulations or measures in one of your studies, you might want to make sure they do not also participate in similar studies. Regardless of the specific reason you want to Exclude workers, there are several easy ways to do so.

How Do I Setup an Exclude List?

You can Exclude workers by specifying studies workers have previously completed, entering a list of Worker IDs, or creating a Worker Group. All three of these options operate by excluding workers who participated in and were approved for your past studies. If, however, you want to exclude workers who may have seen your previous study but not completed it, there are two options for excluding all workers who were associated with your past study. These options are the Super Exclude feature and creating a Survey Group.

 

Finally, if you have a group of workers that have provided sub-par data in the past, you can exclude them from all your future studies by placing them onto a Universal Exclude List.

Excluding Workers Who Have Completed Previous Studies

  1. Exclude Workers from a Previous Study
    To Exclude workers who completed a previous study, simply select which study or studies you want to exclude workers from while on the Worker Requirements page during the study setup.
    Once you are on the Worker Requirements page, first click the Exclude checkbox; then use the dropdown menu to select the study that you would like to exclude.
    knowledgebase_excludingworkers
  2. Exclude Workers Using a List of Worker IDs
    To exclude a list of workers using Worker IDs, paste MTurk Worker ID’s (or CloudResearch ID’s) into the Excluded Workers box on the Worker Requirements page in the study setup.
    knowledegbase_excludingworkers1
  3. Exclude Workers by Creating a Worker Group
    To Exclude workers using a Worker Group, select Create Worker Group from the Worker Requirements page of the study setup.  


knowledgebase_excludingworkers2

On the following page, give your Worker Group a CloudResearch name and choose "Exclude Workers." Then select "Create Worker Group." 
knowledgebase_excludingworkers4

Finally, the last page presents options for managing your worker group. From this page select "Add Workers to Group." On the following page copy and paste your list of Worker ID’s into the text box. Then select "Save." Your worker group should now appear as an option for exclusion while setting up studies.

knowledgebase_excludingworkers5-1

knowledgebase_creatingaworkergroup3

Excluding Workers Who Have Merely Seen Previous Studies

When setting up an Exclude list, there are two options that will ensure all workers are completely unfamiliar with your study. They are the Super Exclude feature and the Survey Group. Both of these features exclude any worker who has accepted specified studies rather than workers who accepted, submitted, and were approved for specified studies like the options above. 

  1. Using the Super Exclude Feature
    To use the Super Exclude feature, first select the Exclude checkbox and choose which study or studies you would like to exclude previous workers from. Then, check the box next to the Super Exclude feature on the Worker Requirements page during study setup. Now, all workers who accepted one of your previous studies will be excluded from your current study.
    knowledegbase_excludingworkers2
  2. Using a Survey Group
    The Survey Groups feature offers a way to exclude workers from multiple studies running simultaneously. Thus, if you or your lab has multiple studies running at the same time and you want to ensure unique workers in each one, you should use the Survey Group feature.
    To create a Survey Group, first click the Exclude checkbox on the Worker Requirements page. From there, select Create Survey Group.

knowledgebase_excludingworkers3


Next, you can give your Survey Group a name and add active surveys to the group. Surveys added to the group will be mutually exclusive once the study goes Live. Finally, select the survey group so that it is active when you launch your studies.

NOTE: The Survey Group feature is intended for launching multiple studies at the same time. Studies can only be added to a survey group while they are being created, not once a study is Live or after it has been Completed.