One of the most controversial issues I have had to deal with since becoming Scientific Editor for the BDJ is that of response rates to questionnaires. The appropriateness of the BDJ's policy, established in January 1997 to reject papers on the basis of specific response rate “thresholds” is a long-standing and contentious issue. Following exhaustive consultation and much soul-searching, my predecessor at the time took a pragmatic view and the principles underlying the leader in which that view was reported (BDJ 1997; 182: 41) remain, in my view, sound, logical and the most appropriate at the time.

In 1997 questionnaire research was viewed by many scientists (including myself) as soft science and an “easy way to a quick publication”. The principles underlying high quality questionnaire research described by Don Dillman in his book The Total Design Method (John Wiley ISBN 0-471-21555-4) were rarely followed by prospective authors. As a result, key scientific principles such as the accuracy of recorded data, the validity of reported results, and trustworthiness of conclusions drawn, were often overlooked and questionnaire surveys developed a poor reputation. This was unfortunate, because many practitioners find such research of great interest and relevance to their day-to-day clinical practice, and it is often the best way to answer certain questions. The BDJ's policy at the time was therefore born out of necessity and has served us well in our desire to remain an international journal that publishes the highest quality scientific research.

In truth, the policy 'guidelines for acceptance of response rates in epidemiological surveys' was simply a way of forcing researchers to put more effort into the design, planning, implementation and reporting of their surveys, but there are now more appropriate ways of achieving the same goal. There remains no evidence base for specific thresholds for response rates in questionnaire studies, but there are core principles based on Dillman's work which appear to have achieved high levels of acceptance within the scientific community and which are now regarded as fundamental to high quality surveys.

I cannot summarise Dillman's book in this leader, but suffice to say, the more work and scientific rigour that goes into the planning and execution of a questionnaire survey, the more likely the results are to be valid. Survey research is no longer an “easy option”. For example, the quality and clarity of the survey's covering letter will improve response rates. The questions set should be simple, short, specific without being too specific, unambiguous and should avoid bias. The questionnaire should be piloted amongst colleagues, potential users of the information and also amongst the population to be surveyed. The questionnaire recipients should be randomly selected, representative of the population as a whole (to avoid selection bias and ensure that results are 'generalisable') and of sufficient size. If one mailing achieves a poor response rate, then repeat the mailing twice and try to identify how non-responders differ (demographically) from responders; there may be underlying and important leasons here.

In my view, very few, if any scientific studies provide definitive answers...it is important to acknowledge the limitations of the research...

In my view, very few, if any scientific studies provide definitive answers, they add to the body of evidence generated from differing studies of differing designs and executed on different populations and in doing so, build up a picture of high probability. Making firm robust conclusions is thus rarely justified and it is important to acknowledge the limitations of the research, so others can repeat it and address some of those limitations (but create others of their own!).

The BDJ decided at its International Editorial Board meeting in Göteborg 2003 to support the principles outlined by my predecessor in 1997 (BDJ, 1997; 182: 68) but to remove the specific thresholds set for response rates and instead to advise all authors and BDJ reviewers to look for Dillman's principles within questionnaire-based research articles. Each paper will be judged on its merit by the expert reviewers, but this represents a toughening rather than a slackening of the guidelines, because simply achieving a high response rate will no longer be sufficient.