What makes a good survey?

In ophthalmology, survey is a common research method used to explore a question or test a hypothesis. A search conducted on February 18, 2023 using “survey” or “questionnaire” as a key word in the journal Eye generated over 1,197 and 799 results, respectively. The journal’s first volume in 1987 contained a manuscript utilizing a survey [1], and such methodology continues to be quite relevant today [2, 3]. In this editorial, the survey process and questionnaire construction are described, followed by discussion of 2 key points to consider when conducting a good survey: sampling and bias.

What is the survey development process?

Fig. 1 presents a concise development process of a good survey that includes: specifying study objectives, developing the survey (writing questions, creating the specific questionnaire), pretesting the survey, data collection from a sample of study subjects, data analysis, data interpretation, and results dissemination.

Fig. 1: Process of a survey.

Development process of a good survey includes: Identifying study objective, preparation, implementation and evaluate results.

How do you construct a survey?

A survey questionnaire should start with a general introduction outlining several parameters including the research team, study objectives, an overview of questionnaire sections, and the approximate time needed to complete the questionnaire. This is followed by questions related to study participants’ sociodemographic and health-related information including age, gender, ethnicity, smoking, alcohol consumption, diet, etc.; disease-specific information such as duration of a condition, severity, family history, and comorbidities; the main body of questions relevant to the survey; and ends with closing remarks (for example, “Thank you for participating in our study!”) [4]. Depending on the study objectives, research teams can choose self-designed questions, adopt a validated patient-reported outcome measure (PROM), [5] or use a combination of both. Some commonly used PROM instruments include the National Eye Institute-Vision Function Questionnaire (NEI-VFQ), Auckland Glaucoma Knowledge Questionnaire, the Impact of Vision Impairment (IVI) questionnaire, and EuroQol-5D (EQ-5D), etc. [6].

One essential step in creating a high-quality survey that we should not forget is to pretest the survey. A pilot study (also called pretest) involves testing the questionnaire content and assessing the flow of a survey process, and this can be performed both within and outside of the research team. This process often involves multiple rounds of testing and refinement of the questionnaire or instrument [7,8,9].

Appropriate sampling

Ideally, the team performing a survey would obtain complete information from every individual in the population of interest to answer a given research question. In reality this is often not possible for a number of reasons. For example, the population of interest can usually be too large, or, simply too difficult to identify the entire population (e.g., patients who had major ophthalmic surgeries in the past five years in a geographic area where the electronic patient records system was new or not established). Therefore, it is often sufficient to study a reasonably selected subset, provided one has a good sampling frame (i.e., complete list) for the population of interest and the sampling is done appropriately.

There are two methods that can be utilized for sampling: non-probability sampling and probability sampling. Non-probability sampling does not ascertain that each individual has a certain probability of being selected and includes methods like convenience sampling (easily accessible participants), purposeful sampling (participants with particular attributes), quota sampling, and snowball sampling (recruited participants recruit further participants). Probability sampling, which is most often used in quantitative survey studies, selects the participants in a random manner that include methods of random sampling, stratified sampling, cluster sampling, systematic sampling, and multistage sampling [7, 9]. We illustrated each level of populations and samples in a survey using a study by Sipkova et al. as an example (Fig. 2). The study objectives were to assess improvement in symptoms and quality of life among adult patients with epiphora 3 months after an oculoplastic surgery [10].

Fig. 2: Levels of Populations and Samples.

Populations and samples in a survey using a prospective study as the example [9]. UK United Kingdom.

When considering the external validity of a study, it is important to ask the question “Outside of this study, would the results be similar in a generalizable population?” Internal validity can be evaluated by considering procedures and factors “inside this study” and asking whether the research had any flaws. In the example above (Fig. 2), external validity could be increased by including more patients who were cared for by additional oculoplastic surgeons in the UK or even in other countries. However, while this approach may improve the external validity, it may simultaneously impair the internal validity, because different surgeons, care facilitators, and different patient populations may lead to different outcomes for many distinct reasons. When designing a survey, it is important to keep the key objective(s) in focus, taking into account feasibility and budgetary restraints.

Survey and data analysis

Common statistical procedures for surveys include descriptive analysis of participants profiles (can be compared with those of the target population to suggest the external validity) and outcomes of interest, as well as inferential statistics that investigate correlation of variables (can be used for hypothesis testing) [7,8,9]. When missing data or major differences between the survey sample and target population pose an important issue, it is important to acknowledge them as a limitation in study and discuss the potential impact that they may bring to the study results. Involvement of a statistician with experiences in a clinical context is always advised [7].

What are possible biases in survey studies?

A bias refers to a systematic error; when possible, these should be recognized and addressed. Biases are common, can impact any study, and can happen in sampling procedures as well as the other survey conduction stages [7,8,9]. For example, in ophthalmic studies, participants who receive a questionnaire and do not respond may disproportionately represent patients who fully recovered from surgery and did not want to spend time completing a survey. In this scenario, patients with poor outcomes may be more likely to complete the survey. A bias in the types of patients who are willing to complete the survey can meaningfully impact the result of the study and can grossly underestimate the benefits of treatment. (Table 1).

Table 1 Possible biases in survey studies.

Surveys can play an important role in acquiring information. Some crucial steps and considerations for conducting a strong survey include appropriate sampling methods to achieve good survey representativeness, pretesting, taking measures to avoid possible biases, and conducting appropriate analysis.