Why is study design important?

A study objective determines broadly who and what is to be studied. The study design — the way in which health status and risk factor data are to be measured and collected and a hypothesis tested — is rarely straightforward. Often there is more than one way to carry out a study.

A badly designed study can lead to erroneous results, or it may not answer the question presented. The design of a study also determines the methods used to analyse the data. It is therefore important to consider the design with a view to how the data will later be analysed. The design of a study can rarely be changed once the study has begun.

What are the main types of study design?

Observational vs experimental Choosing between an observational and experimental study design (Table 1) depends upon the purpose of the study. Observational studies are used to monitor or describe the health status of a population. They are referred to as observational because the investigators merely observe what is happening rather than attempting to intervene in any way. Health data are collected about one or more groups of subjects, from which an inference is made about a target population.

Table 1 Advantages and disadvantages of observational vs experimental designs.

Often measurements of risk factors or exposure data are also collected. These are used to describe the relationship between exposure and the outcome measure of interest, which is often disease or some measure related to disease.

Experimental studies examine the effect of an intervention on the outcome of interest. Comparisons can be made between an intervention and no intervention or between one intervention and another. Health data are collected after and often prior to the intervention, to assess its relative effectiveness.

Prospective vs retrospective The prospective–retrospective dimension (Table 2) describes the way in which data are collected. Prospective studies collect data forwards in time in order to examine the aetiology of disease (observational study) or to assess the effectiveness of an intervention (experimental study). Retrospective studies collect past exposure information on participants through interview or recorded information. Observational studies may be either prospective or retrospective. Experimental studies are always prospective.

Table 2 Advantages and disadvantages of prospective–retrospective designs.

Cross-sectional vs longitudinal Cross-sectional studies give a snapshot of the outcome of interest in a population at any given time and are often used to look at prevalence. Prospective and retrospective studies are examples of longitudinal studies (Table 3). These are used to examine changes in health status over time, including incidence rates of disease outcomes. Repeated cross-sectional studies are sometimes used as a pseudo-longitudinal study, with a different group of participants at each timepoint.

Table 3 Advantages and disadvantages of cross-sectional and longitudinal designs.

The main study types (Figure 1) will be addressed in series of forthcoming publications, addressing cross-sectional, cohort, case–control, ecological and finally randomised controlled trials.

Figure 1
figure 1

Main study types.

Power, reliability and causality

A number of dimensions indicate the quality of a study. The power of a test or a study is defined as the probability of rejecting the null hypothesis when it is false. The greater the power, the better the study is, therefore, at answering the research question, “is there an association between exposure/ intervention and outcome of interest?” when the answer is, “Yes”. In other words, if an association does exist, the power indicates how likely it is that the study will be able to detect this. A more in-depth discussion of power is provided in the final article of the series on randomised controlled trials.

The reliability of a study is a measure of its ability to reproduce the same results if run second or subsequent times. Optimal study design is the key to both a powerful and reliable study.

It is often difficult to make causal inferences between exposures/interventions and outcomes, particularly if based solely on the results of observational studies. Experimental studies are generally better in the assessment of cause and effect because they have better protection from bias and/or confounding (see later articles in this series). There is a standard set of guidelines for causation, first set out by Sir Austin Bradford-Hill in 1965.1

What factors need to be considered when designing a study?

When designing a study, it is important to consider the following points.

First and foremost, what question do you wish to answer?

This should be clearly focussed and ideally written in a form that is amenable to testing (eg, a “null hypothesis” if this is applicable).

Availability of data: obtaining secondary-source data, eg, dental records, will save extra time and cost and may provide a more comprehensive set of results for observational studies.

Sampling methods: the aim is to ensure that the sample is representative of the target population. Inappropriate sampling methods may result in an unrepresentative sample, low response rates or incomplete information, large sample size requirements, contamination and/or bias (see below).

Data collection: the assumption here is that data can be feasibly collected. Data may be unobtainable due to the sensitivity of the study question or the rareness of disease outcome. Where data can be collected, methods include survey by telephone, mail and by face-to-face interview, as well as methods of collection of medical data by health professionals.

Cost of the design, time implications and loss to follow-up: these must be considered, especially when designing a longitudinal study.

Controls: conclusions can be drawn only about the relationship between exposure/intervention and outcome when studies include a control group (ie, they are comparative). Controls may be subjects who have no intervention in a randomised controlled trial or subjects who do not have the outcome of interest in a case–control study. The choice of controls, particularly in an observational study, may not be straightforward (see fifth article in series on case–control studies).

Ethical issues: these arise in response to both the content of the data collected and the methods used in obtaining that data. Particularly in the case of experimental studies, ethical approval of the study will be required. There are potentially considerable ethical implications, where positive interventions exist for one arm of the study and not another.