Main

So you have devised your research question, how are you going to answer it? In this part we discuss the steps to follow when designing your study including the options available for data entry and storage.

Study designs can be divided into two broad categories: qualitative studies and quantitative studies. A study may be purely qualitative, purely quantitative or combine elements of both designs. Qualitative studies, for example, in-depth interviews and focus group work, aim to explore and obtain insight into complex issues, such as the reasons for people's attitudes and behaviours.1,2 The results are described in words rather than numbers. Rather than aspiring to representativeness, qualitative research usually aims to reflect the diversity within a given population.3 The quality of the data is very dependent on the skills of the interviewer. Since qualitative studies tend to be used to explore new topics, they may be used at the start of a quantitative study to identify issues for investigation or as a follow-up to gain insight into unexpected results.

Quantitative studies aim to test a hypothesis. The results are given in numbers or proportions, which most general dental practitioners are familiar with and find easy to conceptualise. The quantitative approach aims to generate data which are representative of the given population. Quantitative techniques are appropriate if the subject is known about, and the research question is relatively simple, clear-cut and amenable to valid and reliable measurement.1 Quantitative study designs may be used to describe how often an event occurs (descriptive study); to examine the relationships between different phenomena (association study); or to compare two or more sample groups (comparative study).

Define study aim

It is very important to define the aims and objectives for your study at the outset of your research. Put simply, the aim of any study is to answer the research question. For example, if your research question is 'Should we send our patients postal reminders?' the aim of the study will be to examine the effect of sending patients postal reminders. At this point it should become clear that you need to be very specific about what exactly you are trying to find out. If you just want to know whether sending reminder cards will increase the attendance rate for dental check-ups at your practice, the aim of your study will be 'To compare the dental attendance rate under the two systems'. Alternatively, if you are also interested in examining the acceptability to patients and the financial costs to the practice of sending reminders, the aim of your study will be 'To examine the acceptability and cost-effectiveness of sending postal reminders to patients'. Whatever you decide, it is important to keep coming back to your stated aim whilst designing your study to avoid going off at a tangent.

Set objectives

Put simply, objectives are the goals that you need to achieve to reach your aim. Objectives should be defined in measurable terms to enable you to assess your progress as the study proceeds. The easiest way to set objectives is to list the steps that need to be taken to answer your question. In the previous example, your objectives might be:

  1. 1

    To compare attendance rates at your practice when patients are sent postal reminders, with attendance rates when no reminders are sent.

  2. 2

    To compare the costs of sending reminders with the costs, in terms of missed appointments, of not sending reminders.

  3. 3

    To survey patients about the acceptability of sending postal reminders.

  4. 4

    To compare the costs and benefits of sending postal reminders to patients.

It is only at this stage that you can decide whether you need a quantitative or qualitative study design or both. In the above example, objectives 1, 2 and 4 are comparative and so you will need to use a quantitative study design. For objective 3 you may choose to use either a quantitative or a qualitative method. An example of a quantitative study design would be to assess the acceptability to your patient population of sending reminders by sending a postal questionnaire to a random sample of patients. Alternatively, you may use a qualitative design and interview a sample of patients, including some who responded to the reminder or, more interestingly, some that did not, to investigate the reasons behind their behaviour.

Quantitative study design

Designing a quantitative study involves making a decision about the overall structure of the study as well as the specific methods to be used. There is a growing evidence base for quantitative research methods. Recent examples include a systematic review showing the importance of concealment of allocation to treatment groups in both non-randomised and randomised trials to prevent an over-estimate of treatment effect,4 and a systematic review showing the measures necessary to improve the response rates to postal questionnaires.5 There are also examples specific to dentistry such as a recent study demonstrating the validity of an abbreviated clinical trial protocol in caries research.6 This evidence matters because it can improve the quality of research which ultimately impacts on clinical care. Thus it is important, wherever possible, to use methods for which there is an evidence base.

It is beyond the scope of this article to address quantitative study design in depth but a brief introduction to some of the important aspects of the process is now given, and a summary is outlined in Figure 1.

Figure 1
figure 1

Quantitative study design stages

Full size image

Hypothesis

The hypothesis is a statement of prediction of the results of the study which the study can then test. In comparative studies, the hypothesis is usually presented in the negative, for example, 'the use of postal reminders will not affect patient attendance rate.' This is known as the null hypothesis and the study seeks to disprove it.

Type of study

The type of study design chosen is determined by the research aims. Studies may be experimental or non-experimental. An experimental study aims to control all relevant variables whilst altering only the variables under investigation. Such studies are often used to test the effectiveness of an intervention, for example the effect of fissure sealing on caries rate. The randomised controlled trial is the gold standard design for this type of study.

Non-experimental studies involve the researcher gathering data without intervening to control variables. Longitudinal cohort studies, case-control studies and cross-sectional surveys are examples of this type of study design. Non-experimental research in general dental practice is most likely to involve the use of a cross-sectional survey. A cross-sectional survey is a system for collecting data at a single time point, although the data may refer to health experiences in the past. Cross-sectional surveys can be used to answer descriptive questions such as the factors influencing dentists' choice of restorative materials. They can also address questions of relationship, for example, the relationship between anxiety and dental treatment history, and questions of comparison such as the difference in the prevalence of periodontal disease between two groups of patients.

Choosing the variables to be measured

A variable is something which is measured in a study, for example, the participants will have various demographic variables (sex, age etc), and measurements will be taken of other variables (a patient's level of anxiety, satisfaction with treatment etc). The dependent variable, for example, the patient's satisfaction with treatment, is assumed to depend on the independent variable, for example the patient's age. When a relationship between a dependent and an independent variable has been observed, it is always possible that a different independent variable has produced the effect. This variable is called a confounding variable.

Study setting and inclusion/exclusion criteria

The setting for a study is the population to which the results can be applied. For example, if you limit your study to patients from your own practice then, strictly, the results can only be applied to your practice. Moreover, if you limit the study to a 1-month period then, unless you can demonstrate that the month you have chosen is typical of the year as a whole, the results of the study can only truly be applied to that particular month.

The study population should be defined at the outset, together with the characteristics of the subjects that you wish to include (inclusion criteria). For example, we may define the population for the postal reminders/dental attendance study mentioned above as all the adult patients who have been registered with the practice during the last two years. You should also define the characteristics of subjects who should not be included (exclusion criteria). In the above study, for example, you may decide to exclude patients who did not finish a course of treatment. It is not good practice to decide to exclude participants for spurious reasons as you go along. Inevitably, there will be some subjects for whom adequate data cannot be collected. The best way to approach this problem is not to pretend that these subjects did not exist, but to record as much detail about them as possible. At the end of the study you can then compare the characteristics of participants and non-participants and discuss any major differences between the two groups in your study report (see Part 6 in this series on data analysis).

Sampling

Sampling provides a means of obtaining representative information about a population without having to investigate all the members of the population. Where a list of the population members is available, this is known as the sampling frame, for example a list of patients, or the electoral register. There are several sampling techniques available but for all quantitative studies, the fundamental principle is to include a sample of subjects that are representative of the population that you wish to apply your study results to. This is often done by random sampling, a form of sampling in which every member of the target population has an equal chance of being included. There are published random number tables7 and computer software available that you can use to generate a list of random numbers to indicate which subjects to include in your sample.

Sample size

In the past, determining sample sizes was an inexact science often based on round numbers such as 30 and 100. Now computer programs can calculate exactly the number of subjects to include in your study to ensure that your findings have not merely occurred by chance. This is the time in the study design process when you should consult a statistician (see Table 1 for contact details). It is not acceptable to collect all your data first and then visit your local statistician and ask 'is this result significant?'

Table 1 Further information
Full size table

A statistician will require certain information in order to help you. For nominal data (see later) you should expect to inform the statistician how often a particular outcome is likely to occur. You should have also decided what size of effect or difference is clinically significant. For interval data (see later) the statistician will need some indication of the mean and standard deviation of the control group in a comparative study (see Part 6 of this series). This might be obtained, for example, from a pilot study or by examining patient records. The level of significance to be used (eg P ≤ 0.05) and the power of the study to detect an important difference at the given level of significance, will also be discussed.

Since the sample size that you will be given will be exact, it is prudent to increase the number of subjects in your design to allow for non-respondents or participants who drop out of the study. If you are unable to amass a large enough sample from your own practice then it is worth considering teaming up with other practices. Be aware however, that 'practice factors' may also affect the results and so the sample size will need to be recalculated depending on the number of practices involved. A study involving more than one practice requires a higher level of organisation than a single practice study, and the time and cost of recruiting and maintaining the motivation of participating dentists should be considered carefully.8

Control groups in comparative studies

A decision must be made as to what constitutes a suitable control treatment. It may be more informative to compare a new intervention to the best treatment currently available, than to compare it with a placebo. In a parallel group study design, one group receives the intervention treatment at the same time as the other group receives the control treatment. In a crossover design, each participant receives both the intervention and control treatment, separated by a 'washout' period of no treatment, if applicable. In both designs, there should be random allocation of the participants to the intervention and control treatments.

Recruitment of participants

The recruitment of participants from the study population will depend on what sort of study is being undertaken. In a clinical trial, patients may be invited to participate in the study as they attend the surgery (a consecutive sample). In surveys, the subject is invited to participate in person, through a letter, or by telephone. Motivating subjects to participate in your study involves establishing trust, demonstrating the importance and relevance of the study, minimising costs and providing rewards.9 The use of payments, inducements, or free treatment to attract patients to take part may be considered but raises a number of ethical considerations, particularly when dealing with vulnerable groups.5 As with all studies, the participants should receive adequate written information about the study before giving written consent to participate. The written information should have been approved by your local research ethics committee (LREC) before the commencement of study (see Part 6 of the series).

Blinding

The importance of concealment of allocation to treatment groups in both non-randomised and randomised trials to prevent an over-estimate of treatment effect has already been mentioned.4 Ideally, a clinical trial should be double blind where neither the investigators nor the subjects know which treatment has been received. This may not be practical, but as a minimum, the trial should be single blind where the person making the assessment, for example, the patient completing a pain questionnaire or the pathologist interpreting the diagnostic specimen, does not know which intervention has been given.

Outcomes measures

You also need to decide at this stage in the design process how you are going to measure the outcomes for your study and the tools that you are going to use to record your results. It is also important to state in this part of the study design how all the researchers will be trained in the use of the measures and the procedures that you will use to test that the data recorders are reliable. These issues will be discussed in the next part in this series. In clinical trials, it is important to establish end-points. These are specified outcomes on the basis of which the hypothesis is accepted or rejected. An end-point may be a clinical response, for example, the proportion of patients achieving a 50% improvement in periodontal bleeding score over the 12-week treatment period, or a quality of life measure, such a the proportion of patients achieving a 75% improvement on a specified pain index after a 4-week treatment period.

Study plan

All study procedures should be carefully planned and documented. The order, timing, frequency and site of all study procedures should be detailed, as well as drugs given, dosages, route of administration, equipment to be used etc. The responsibilities of each member of the research team should be specified. Data collection sheets, patient information sheets, consent forms etc. should be written and approved by the LREC. You should expect to have to make minor modifications to the plan after the study has been piloted as problems may become apparent.

Bias

There are many threats to the reliability (repeatability) and validity (closeness to the truth) of a study. These are known as biases and such errors can creep in at every stage of the study from the conceptualisation of the research idea, through the design stages, sampling, data collection, and analysis stages of the study.1 They can affect all the people involved in the study, not just the researchers, for example, recall bias relates to respondents' selective memories in recalling past events or experiences. It is important to be aware of potential biases and make every effort to reduce sources of error.

Hazards, risks and problems

Careful planning and anticipation of problems and hazards can ensure that the study runs smoothly. It is especially important to have a clear strategy for dealing with patients who develop adverse reactions during a clinical trial. The potential costs arising from such a problem should also be considered.

Data entry and storage

Practical issues

Practical considerations include the coding of data (how codes are assigned to values), the manner in which data are stored (the program used and the format of files), and the compatibility of data storage and data analysis programs. You will also need to decide whether the results should be recorded on paper and transferred later to a spreadsheet or database, or entered directly into a computer.

It is advisable to discuss data storage and the data analysis strategy with an experienced researcher. Data analysis is covered in Part 6 of this series. Your data file can take the form of a spreadsheet with individual people forming the rows of the spreadsheet, and the variables forming columns (Fig. 2). A spreadsheet program such as Microsoft Excel© is a good and simple way to enter data. Alternatively, you may wish to enter data directly into a database. Microsoft Access© is a good program to use if you intend to generate a large number of contacts with patients, for example, in a questionnaire survey. Lists of patients to contact can be generated within Microsoft Access© and then mail-merged with letters to patients stored in Microsoft Word©. Undertaking any form of statistical analysis in Microsoft Access© is difficult however, and it is easier to transfer the data to a statistical package such as SPSS for the analysis.

Figure 2
figure 2

Example of a spreadsheet

Full size image

In general, the simple rule to follow with data entry is to have a single variable for each item of information which you may wish to analyse. It is important to distinguish between different types of data variables. Variables can be:

  • Nominal (categorical) — cases are divided into groups but there is no implied order in the groups eg gender, sex, Angle's Orthodontic Classification.

  • Ordinal (ordered categorical) — cases are divided into groups which have an implied order eg IOTN categories,10 age groups, Lickert scales.

  • Interval (continuous) - items are measured on an ordered scale with equal intervals between the points on the scale eg age measured in years, number of carious teeth.

When entering quantitative data, numerical codes should be assigned to the value of nominal and ordinal variables to simplify data entry and facilitate statistical analysis.11 So, for example, gender (nominal variable) can be coded 1 = male, 2 = female, and age groups (ordinal variable) can be coded 1 = 0–10 years, 2 = 11–20 years, 3 = 21–30 years etc. Interval variables are already in a number form and therefore do not have to be coded.

There will inevitably be some cases where data are missing. It is recommended that rather than leaving a coding box blank, you allocate a specific value such as '9' for missing data. You will then know for sure that the data were missing.

Legal issues surrounding data collection and storage

Most of the legal issues about storing research data arise out of the Data Protection Act.12 The Act places restrictions on the storage of any data in which the individual could be identified. Furthermore, it places restrictions on the analysis of data for purposes other than that for which it was originally intended. If you are storing patient information on a dental practice computer then you should have already registered with the local Data Protection Officer. The research project, however, must be registered separately because the data are being used for a different purpose. Access to the data should be password-protected and limited to individuals involved in the data collection process. It should not be possible to trace data back to individual participants. The use of unique subject ID codes to identify participants, rather than their names, overcomes this problem of confidentiality.

Qualitative study design

Qualitative studies aim to explore, interpret or obtain a deeper understanding of little known or complex issues. This section of the paper briefly introduces some of the important aspects of qualitative study design. Qualitative data can be obtained by various methods. For example, the in-depth interview is a face to face conversation between the researcher and the participant which allows exploration of issues or topics of relevance. Another method is the focus group where the researcher facilitates a group discussion in which group interaction generates the data.

Sampling

It is important to appreciate that the sampling methods used for qualitative studies are very different from quantitative studies which aspire to statistical generalisability. Qualitative research usually aims to reflect the diversity within a given population. There are several sampling techniques available but the most common is purposive sampling whereby participants are chosen because of their particular demographic characteristics, behaviours, attitudes or experiences. For further details on sampling methods for qualitative studies see Silverman.13

Sample size

Depending on the research aims and the study population, a decision has to be made as to how many people will be recruited to the study. The number of participants is usually smaller than in a quantitative study but the information received from each participant is of greater depth. You should decide at the outset what groups or types of people will be interviewed. However, since the aim of qualitative research is to explore and understand rather than quantify, the number of participants is often determined by issues that emerge from the research process such as the range of experiences described or the variety of different explanations given. Efforts should be made to obtain data that contradict explanations or information already received by extending the sample into a different area or group.14

Data collection

The process of data collection for qualitative studies is very dependent on the skills of the interviewer or focus group facilitator. The interviewer is usually guided by a topic list and asks open-ended questions to encourage respondents to tell their story in their own words. Since it is very easy to influence the responses that are given, interviews should only be undertaken by trained researchers (see Table 1 for further information on training courses). Even so, it is not possible to fully control for the researcher's perspective or interviewer bias and thus it is important to clearly state the background or perspectives of the researcher in the study report. All interviews and focus group discussions should be recorded on tape and if appropriate, supplementary written 'field notes' taken.1

Storing qualitative data

Qualitative data consist of large amounts of tape recorded or written information. The essence of data storage is to get all information into written transcripts which are then analysed. To conform with the Data Protection Act, the transcripts need to be anonymised as before. The tapes should be destroyed at the end of the study. Figure 3 gives an example of a transcript from a focus group interview.

Figure 3
figure 3

Example of a focus group transcript

Full size image

Summary

In this article we have briefly described the steps that you should follow in designing a study to answer a research question. The importance of defining the aims and objectives of the study at the outset and continually referring back to these during the study design process has been emphasised. We have discussed some differences in approach and design of qualitative and quantitative studies. We have also stressed the need to seek advice from an experienced researcher and/or statistician at an early stage to ensure that the data collection process is as effective and efficient as possible.