Study Design

A good study design can prevent or limit problems with confounding, measurement error, selection bias, which would threaten the validity of the results and limit generalizability. Statistical analysis cannot compensate for serious design mistakes. 

The appropriateness of a design depends on the research question, availability of measurement tools, and how the information will be used. The validity of a design is highly topic- and context-specific. There are competing considerations, for example, when a measurement instrument is more precise, but also more expensive, thus necessitating a smaller sample size due to budget constraints. Or, there are laboratory measurements that are kwon to be better predictors than others, but have more missing values.

Step 1. Define the study question.

• Formulate your study question in one sentence
• To get help
  • Check out published study protocols
  • Focus your research question by limiting the scope (inclusion/exclusion cirteria)
  • Figure out what you might expect.
  • What has already been done in the literature?
• Review literature on the topic.  Is the study redundant?  Will it contribute to the understanding of the topic? Will your study advance the science in its field? 
• Summarize literature on the topic. What was the sample size? What was the time frame? What were the subgroups? What were their outcomes? What were the results? How is your study different from these?
• Typically, the initial idea or study question is too broad.  Focus. Focus. Focus. Break it down into smaller testable hypotheses.
• What variables are important in answering your research questions? What measurement instruments do you have available? Are there potential issues with using these variables (missing values, confounding, etc)?
• Important! Make sure you have enough time and resources to complete the study in a timely manner.

Step 3.  Propose the study question to PI/advisor/mentor/collaborators

• Set up an appointment.  
• Bring examples or related studies. Be prepared to articulate and defend your idea.
• Be open to ideas and suggestions. 

Step 4.  Feasibility assessment.

• Determine who will helpwith the project. 
  • Statistician 
  • Domain expert 
  • Data management – Some projects might require assistance with database set up, construction of metadata, data cleaning, or further data management (ask CSTAT, if you need help)
• Do you have a large enough sample sizeto make a meaningful study?  Sufficient power? 
  • Ask your PI/advisor/mentor/collaborators, “Do you think we have/can get a large enough sample for this study?”
  • Obtain a preliminary estimate of sample size using statistical software packages.
• Estimate time required to gather, analyze, and publish data.  Create a study timeline. Be aware that most studies will take a year or more to complete and publish.
• Do you have incomplete data? If you are using imaging or pathology specimens, what percent are missing?  If a key aspect of your study is requires a certain variable and only a small percentage of your population will have that information you need to think about the implications, adjust your study design, sample size, and analysis plan and figure out whether you can account for this bias. 

Step 5.  Establish the study design and study protocol

Nobody expects you to be an expert in study design. Know the basics, rely on the expertise of your statistician. Statisticians can help to make sure that the statistical models indeed answer the research question. Statisticians can suggest more complex models you may not have thought of, that may lead to a revision of your research questions. Remember, the more time you invest upfront in the set-up of your protocol/study design, the more time, frustration and money you will save down the road.

• Take a course on study design?
• Define study design:  case report, cohort, case-control, nested case-control, cross-sectional, interventional, ….
• Define type of data required for study: 
  • electronic records
  • biospecimens
  • imaging
  • laboratory measurements
  • Survey – CSTAT is well equipped to design surveys using Qualtrics and MSU has the Office for Survey Research Center  Note: You need to factor the time of developing the survey, mailing it, waiting for responses and follow-up into your study timeline.
• Establish the scope of your study. Define inclusion and exclusion criteria. This will help readers understand, if your study is generalizable and where/to whom it would apply.
• Request and meet with a statistician. 
  • Submit request to meet with a statistician:  http://cstat.msu.edu
  • A statistician can help write the “statistical methods section.”
  • A statistician can help determining the appropriate sample size.  
  • Prepare for the meeting!!