Setting up a Longitudinal Study

Longitudinal studies, also sometimes known as tracker or follow-up studies, sample from the same group of participants over time. To set up a longitudinal study with the MTurk Toolkit, follow the instructions below.

Setting Up Your Initial Study

To set up a longitudinal study, select Create a Study from the header on the CloudResearch homepage and choose the MTurk Toolkit.

On the next page, set up your study as you normally would.

 

TIP! To help minimize attrition, it helps to recruit workers who are committed to multiple waves of data collection from the start of the study. Hence, we recommend when writing the Description and Custom Instructions for your HIT, that you explicitly state the study contains multiple waves of data collection. Also, in the description area, describe the payment plan for the entire study (e.g., “You will be paid $1 for the initial study and $5 if you complete the second part of this study.”) Higher payment (at least $8 per hour) and a more transparent process (e.g., “You will be re-contacted in three weeks”) will lower attrition.

 

study description

 

NOTE: While setting up your initial study, it is imperative that you collect participants’ MTurk Worker IDs. Without Worker IDs, there will be no way to match participants’ data across time points. If you are worried about worker anonymity, CloudResearch can anonymize Worker ID's, allowing you to use the encrypted CloudResearch ID just as you would an MTurk ID, while giving workers an extra layer of privacy.

To collect Worker IDs you can: a) create a question within your survey that asks workers to enter their ID or b) use embedded data to automatically capture each Worker’s ID as they enter the survey.

Setting Up a Follow-Up Study

When running a follow-up study or successive waves of data collection, the Include Workers and Email Workers features are critical.

 

To set up a follow-up study, begin by creating a new study with the MTurk Toolkit. When you reach the Worker Requirements page, use the Include feature to specify the eligible workers. Then, after you launch your study, use the Email Workers tool to email workers on your Include list and let the know the study is available.

Include Feature

CloudResearch’s Include feature is fundamental to running longitudinal studies on MTurk. The Include feature allows you to specify which workers are eligible for the study; only workers you place on the list are able to take your survey.

 

The Include feature is located on the "Worker Requirements" page in the study setup. There are two ways to use the Include Feature:

  1. At the study level by selecting previous surveys to include.

    includesurvey2
    This method will only include workers who have completed All or Any of the surveys specified.
    include all surveys

    When All is selected, workers who have completed every study specified will be included. Think of this as an AND feature; workers who have completed survey 1 AND survey 2 AND survey 3 will be included in the current survey.
    When Any is selected, workers who have participated in any one of the surveys specified will be included. Think of this as an OR feature; workers who have completed survey 1 OR survey 2 OR survey 3 will be included in the current survey.
  2. At the worker level by manually entering MTurk Worker IDs.
    To manually include workers, you will need to retrieve Worker IDs from a previous study. On your study Dashboard, click on the study you want to obtain Worker IDs from and select “Download CSV” on the right side of the screen.
    csvdownload
    A .csv file will download from your web browser. Open the file and copy the MTurk IDs from the first column, AmazonIdentifier.
    csv dataNext, paste the MTurk IDs into the manual include textbox on the Worker Requirements page of the study setup.
    include-workers-by-worker-idsNOTE: Only workers who have completed one of your past studies AND been approved for that study may be included when using the Include Feature.

Letting Workers Know the Follow-Up Study is Available

Once your follow up is ready to go, you can use CloudResearch’s Email tool to quickly and easily let all eligible workers know that the follow up is available.

 

To use the Email tool, go to your follow-up study on the Dashboard and select the orange "Email Included Workers" button. The Email tool only appears on studies that use an Include list.

 

email-included-workers

 

Once you select "Email Included Workers," you will be taken to a page where you can draft a message to send to all workers. The default Subject includes the compensation of the HIT, how long on average it has taken participants to finish, and a link to the HIT on MTurk.

 

NOTE: When customizing your message, do not delete the link to the HIT. Without this link, workers will be unable to navigate to your study.

email included workers page

 

For more information on running longitudinal studies, see our blog or knowledge base article with tips for running longitudinal studies.