Longitudinal studies, also sometimes known as tracker or follow-up studies, sample from the same group of participants over time. To set up a longitudinal study with the MTurk Toolkit, follow the instructions below.
Setting Up Your Initial Study
To set up a longitudinal study, select Create a Study from the header on the CloudResearch homepage and choose the MTurk Toolkit.
On the next page, set up your study as you normally would.
TIP! To help minimize attrition, it helps to recruit workers who are committed to multiple waves of data collection from the start of the study. Hence, we recommend when writing the Description and Custom Instructions for your HIT, that you explicitly state that the study contains multiple waves of data collection. Doing so is one way to minimize attrition. Also, in the description area, describe the payment plan for the entire study (e.g., “You will be paid $1 for the initial study and $5 if you complete the second part of this study.” or “You will be paid $6.00 for the entire study.”) Higher payment (at least $8 per hour) and a more transparent process (e.g., “You will be re-contacted in three weeks”) will result in lower attrition.
NOTE: While setting up your initial study, it is imperative that you collect participants’ MTurk Worker IDs. Without Worker IDs, there will be no way to match participants’ data across time points. If you are worried about worker anonymity, CloudResearch can anonymize Worker ID's, allowing you to use the encrypted CloudResearch ID just as you would an MTurk ID, while giving workers an extra layer of privacy.
To collect Worker IDs you can: a) create a question within your survey that asks workers to enter their ID or b) use embedded data to automatically capture each Worker’s ID as they enter the survey.
Setting Up a Follow-Up Study
When running a follow-up study or successive waves of data collection, the Include Workers and Email Workers features are critical.
To set up a follow-up study, begin by creating a new study with the MTurk Toolkit. When you reach Tab 7 - the Worker Requirements page, use the Include feature to specify the eligible workers. Then, after you launch your study, use the Email Workers tool to email workers on your Include list and let the know the study is available.
Include Feature
CloudResearch’s Include feature is fundamental to running longitudinal studies on MTurk. The Include feature allows you to specify which workers are eligible for the study; only workers you place on the list are able to take your survey.
The Include feature is located in Tab 7 - the "Worker Requirements" page in the study setup. Firstly, select the ‘include’ option. Then, you have two options for inclusion: by survey or by worker Id.
- Include by Survey - from the dropdown menu select the survey(s) that you would like to include. Ex. for a follow-up study you can choose to include anyone from prior iterations of the study
When All is selected, workers who have completed every study specified will be included. Think of this as an AND feature; workers who have completed survey 1 AND survey 2 AND survey 3 will be included in the current survey.
When Any is selected, workers who have participated in any one of the surveys specified will be included. Think of this as an OR feature; workers who have completed survey 1 OR survey 2 OR survey 3 will be included in the current survey.
- Include by worker Id - in the textbox enter each individual MTurk Worker IDs, separating each ID with a line break.
Often it is helpful to retrieve Worker IDs from a previous study on CloudResearch. To do this, find the study on your study Dashboard and click on options. Then, from the dropdown menu, select “Download Amazon IDs/Anonymized CSV”.
Next, paste the MTurk IDs into the manual include textbox on the Worker Requirements page of the study setup.
NOTE: Only workers who have completed one of your past studies AND been approved for that study may be included when using the Include Feature.
Letting Workers Know the Follow-Up Study is Available
Once your follow up study is ready to go, you can use CloudResearch’s Email tool to quickly and easily let all eligible workers know that the follow up study is available.
To use the email tool, find your study on the dashboard and click options. Then, from the dropdown menu select ‘Email Included Workers.’
Once you select "Email Included Workers," you will be taken to a page where you can draft a message to send to all workers. The default Subject includes the compensation of the HIT, how long on average it has taken participants to finish, and a link to the HIT on MTurk.
NOTE: When customizing your message, do not delete the link to the HIT. Without this link, workers will be unable to navigate to your study.
For more information on running longitudinal studies, see our blog or knowledge base article with tips for running longitudinal studies.