Setting up MTurk Toolkit Studies
      Back to home
      1. Knowledge Base
      2. Designing and Launching Studies
      3. Setting up MTurk Toolkit Studies

      Setting up the HIT and Payment

      There are several important decisions to make on the Setup HIT page including deciding how much to pay participants, how long to allow people to work on your study, and whether to batch your study among others.

      Participant Payment

      One of the most important details of your study is deciding how much to pay the people who complete it. The amount of money you offer may affect everything from how fast the data are collected, to the percentage of experienced participants in your data set, to the attrition rate in longitudinal studies. When deciding how much to pay participants, we recommend considering a few factors.

      1. As a rule of thumb, you should pay a minimum of $0.10 per minute or $6 per hour.
      2. If your study requires more from participants than answering survey questions, you should pay more. Dyadic studies, longitudinal studies, studies with lots of writing, and anything that requires participants to download an app or program, should all pay more.
      3. The most accurate way to determine how long your study takes, and therefore how much you should offer as compensation, is to run a pilot study.

      To enter the payment amount, simply click the Worker Payment Per Survey box and type the amount you wish to pay.

      knowledgebase_setuphit

      Expected Time to Complete HIT

      You can communicate how long you expect the HIT to take in the “Expected Time to Complete HIT” box. The number you enter here will be added to your HIT description page on MTurk. Participants often decide whether to take a HIT or not by considering how much the HIT pays and how long it is expected to take.

      Time Allotted Per Assignment

      The "Time allotted per assignment" is the maximum amount of time a participant has to complete your HIT. If a participant does not submit the HIT within the time allotted, they will be locked out and not paid. For this reason, you should make sure participants have enough time to complete your study. 

      At the same time, however, giving people too much time (e.g., 24 hours) can be problematic because some participants may not complete the study in one sitting, may accept the HIT but wait until later to complete it, or for technical reasons the HIT may be unavailable for a long period of time after a participant abandons it.

      To balance these concerns, we recommend setting the time allotted per assignment to three or four times the expected length of the survey. If the survey is expected to take 15 minutes, we recommend giving participants 45 or 60 minutes to complete it.

      To enter the Time Allotted per Assignment, simply click on the box and type in the number of minutes, hours, or days, you want to give participants to work on your HIT. Please note that if your time allotted per assignment is greater than 90 minutes, the restart function will not be available.

      knowledgebase_setuphit1

      HIT Expiration

      The HIT expiration box allows you to specify how long you want your HIT to remain open and visible on Mechanical Turk. The default setting is 30 days. If you would like to shorten or lengthen this time, simply click on the box and enter the number of days, hours, or minutes, you would like your HIT to be active for.

      Automatic Bonus

      If you're running a study in which all participants who complete the study receive a bonus, you can schedule an Automatic Bonus to be awarded as participants are approved for your study. To do so, just add the amount you want to bonus each person into the Automatic Worker Bonus box and your bonus will be set. Participants will be given a bonus in addition to whatever amount you specify as base compensation.

      knowledgebase_setuphit2

      Batching Options

      • MicroBatch

      MicroBatching breaks your study into multiple HITs of 9 assignments or less and runs HITs sequentially, allowing you to slow down data collection. MicroBatch cuts MTurk fees from 40% to 20%.

      • HyperBatch

      HyperBatching breaks your study into multiple HITs of 9 assignments or less and runs all HITs simultaneously. HyperBatching is the fastest way to collect data and cuts MTurk fees from 40% to 20%.

      To learn more about the difference between MicroBatch and Hyperbatch see here.

      NOTE: MicroBatch and HyperBatch are mutually exclusive; selecting MicroBatch disables HyperBatch and vice-versa. 

      Additional HIT Options

      Finally, the last thing to decide on the HIT Setup page are a few details about how you want your HIT to appear to participants.

      1. You can decide whether participants who are not qualified to take your HIT should be able to preview the HIT or not. Limiting the HITs' visibility to only those who are qualified is necessary in order to accurately measure the Bounce rate.
      2. You can decide whether to display the Median Hourly Wage on your HIT. Advertising the median hourly wage to participants is something CloudResearch offers in order to help participants decide which HITs to work on. If your study pays a competitive wage, turning this option on can make your HIT more attractive.
      3. Finally, you can decide whether to allow CloudResearch to add a few demographic questions to your HIT submission page. These demographic questions are randomly selected, voluntary for participants, and how CloudResearch determines participant demographic characteristics for the purpose of creating Panel options. When you allow CloudResearch to add demographic questions to your HIT, each participants’ gender consistency score will be automatically added to your CloudResearch csv file.

        knowledgebase_setuphit3