There are several important decisions to make on the setting up your HIT and payment tab including deciding how much to pay workers, how long to allow workers to work on your study, and whether to schedule a launch time among others.
One of the most important details of your study is deciding how much to pay the people who complete it. The amount of money you offer for your study will affect everything from how fast the data are collected, to the percentage of experienced workers in your data set, to the attrition rate in longitudinal studies. When deciding how much to pay workers, we recommend considering a few factors.
- As a rule of thumb, you should pay a minimum of $0.10 per minute or $6 per hour.
- If your study requires more from workers than answering survey questions, you should pay more. Dyadic studies, longitudinal studies, studies with lots of writing, and anything that requires workers to download an app or program, should all pay more.
- The most accurate way to determine how long your study takes, and therefore how much you should offer as compensation, is to run a pilot study.
To enter the payment amount, simply click the Worker Payment Per Survey box and type the amount you wish to pay.
Expected Time to Complete HIT
You can communicate to workers how long you expect the HIT to take. The amount of time you enter in the Expected Time to Complete HIT box will be added to your HIT description page on MTurk. Workers often decide whether to take a HIT or not by considering how much the HIT pays and how long it is expected to take.
Time Allotted Per Assignment
The "Time allotted per assignment" is the maximum amount of time a participant has to complete your HIT. If a participant does not submit the HIT within the time allotted, they will be locked out and not paid. For this reason, you should make sure workers have enough time to complete your study. At the same time, however, giving workers too much time (e.g., 24 hours) can be problematic because some workers may not complete the study in one sitting, may accept the HIT but wait until later to complete it, or for technical reasons the HIT may be unavailable for a long period of time after a worker abandons it.
To balance these concerns, we recommend setting the time allotted per assignment to three or four times the expected length of the survey. If the survey is expected to take 15 minutes, for example, you should give workers 45 or 60 minutes to complete it.
To enter the Time Allotted per Assignment, simply click on the box and type in the number of minutes, hours, or days, you want to give workers to work on your HIT. Please note that if your time allotted per assignment is greater than 90 minutes, the restart function will not be available.
The HIT expiration box allows you to specify how long you want your HIT to remain active and visible on Mechanical Turk. The default setting is 30 days. If you would like to shorten or lengthen this time, simply click on the box and enter the number of days, hours, or minutes, you would like your HIT to be active for.
If you created your study with an external survey creation platform like Qualtrics or Alchemer, you can enter the link to your study in the Survey Hyperlink box. Once your link is pasted into the box, you can preview the study by clicking the "Preview" button.
If you are setting up your study in advance and do not have your survey link yet, you can enter any URL as a placeholder (must contain http://www.). You cannot save a HIT without a URL.
Schedule a Launch Time
You can schedule a time for your HIT to launch using the Scheduled Launch Time box. Choose a date and time in the future when you would like for your HIT to be automatically launched and enter it into the box. The HIT launch time must be scheduled in Coordinated Universal Time (UTC), so be sure to make the conversion to your local time.
Once you schedule a launch time and save your study, you can select the “Launch” button to activate the launch time. Your study will stay in a starting state until the set launch time is reached, at which point the study will go live. While in the starting state, your study may appear stuck with a continuously spinning wheel—this is normal. The study will stay in this state until the scheduled launch time is reached.
Finally, it is important to note that when your scheduled launch time is reached, the study will begin to go live, but may not be live right away. As with any HIT, the amount of time it takes to actually go live after the launch button is selected depends on the number of qualifications MTurk needs to assign for your study. Qualifications are primarily determined by the number of inclusion and exclusion criteria associated with your study (e.g., an exclude list or demographic targeting).
If you're running a study in which all workers who complete the study receive a bonus, you can schedule an Automatic Bonus to be awarded as workers are approved for your study. To do so, just add the amount you want to bonus each person into the Automatic Worker Bonus box and your bonus will be set. Workers will be given a bonus in addition to whatever amount you specify as base compensation.
Finally, the last thing to decide on the HIT Setup page are a few details about how you want your HIT to appear to workers.
- You can decide whether workers who are not qualified to take your HIT should be able to preview the HIT or not. Limiting the HITs' visibility to only those who are qualified is necessary in order to accurately measure the Bounce rate.
- You can decide whether to display the Median Hourly Wage on your HIT. Advertising the median hourly wage to workers is something CloudResearch offers in order to help workers decide which HITs to work on. If your study pays a competitive wage, turning this option on can make your HIT more attractive to workers.
- Finally, you can decide whether to allow CloudResearch to add a few demographic questions to your HIT submission page. These demographic questions are randomly selected, voluntary for workers, and how CloudResearch determines worker demographic characteristics for the purpose of creating Panel options. When you allow CloudResearch to add demographic questions to your HIT, each worker's gender consistency score will be automatically added to your CloudResearch csv file.