Quality measures for dental care: A systematic review

Abstract

Data sources MEDLINE, EMBASE and LILACS were searched. Reference lists of the included publications were screened to identify other potentially relevant documents.

Study selection Articles published from 1 January 2002 to 31 December 2017. There was no language restriction for full-text articles as long as they had a title, abstract and description of the quality measures in English. Publications that either described the development process or described the clinimetric properties of oral healthcare quality measures for general dental care were included. Publications were only included if numerators and denominators of the quality measures were defined or could be directly derived. Editorials, randomised controlled trials, conference abstracts and letters to the editor were excluded.

Data extraction and synthesis Two researchers independently screened the titles and abstracts. Information included study purpose, the country of origin, methods used to develop measures and stakeholder involvement. The data considered included the number of quality measures developed, the description, numerators and denominators of the measures and the type of quality measure as described by Donabedian ie process, structure or outcome measure. The Appraisal of Indicators through Research and Evaluation (AIRE) instrument 2.0 was then used to appraise the scientific soundness and applicability of the measures

Results Twenty-four publications were included yielding 215 quality measures. AIRE scores ranged from 38 to 78 out of 80 possible points. The majority of measures (n = 71) referred to treatment and preventive services. Comparably, few measures referred to the domain patient safety (n = 3). The development process of measures often exhibited a lack of involvement of patients and dental professionals. Few projects reported on the validity (n = 2) and reliability (n = 3) of the measures. Four projects piloted the measures for implementation in practice.

Conclusions Provides an overview of existing quality measures in oral healthcare. Potential opportunities include the piloting and testing of quality measures and transparent data reporting on the quality of oral healthcare.

A Commentary on

Righolt A J, Sidorenkov G, Faggion C M, Listl S, Duijster D.

Quality measures for dental care: A systematic review. Community Dent Oral Epidemiol 2019; 47: 12-23.

GRADE rating

Commentary

Data collected by healthcare systems continues to increase. There is little consensus around valid and reliable measures of clinical quality . Minimum prerequisites for measuring quality are suggested as: being based on scientific evidence; accepted by experts in the field; and measured using reliable data sources.¹ The extent to which these prerequisites are met by quality measures for oral health in commissioned dental services in the UK alone is questionable. This systematic review aimed to provide an overview of quality measures in oral healthcare and to 'evaluate the scientific soundness and applicability of these quality measures'.²

Search strategies and methods are clear and comprehensive.

The AIRE instrument 2.0 used contains 20 criteria divided into four domains: (1) purpose, relevance and organisational context; (2) stakeholder involvement; (3) scientific methods; and (4) additional evidence, formulation and usage. Each AIRE item is scored on a four-point Likert scale ranging from 1 to 4 (where 4 = confident that the criterion has been fulfilled).

For anyone developing (or assessing) quality measures for dental care, the review is useful in highlighting the importance of combining stakeholder views, clinical evidence and an assessment of the reliability and validity during development of the measures. The 215 yielded measures are given in Appendix S5 and this forms a valuable collation of quality measures in dental care. The authors recommend four publications as scoring highest on the development, testing and validation of the measures.

Looking at the actual measures used in some of these high-scoring publications highlights an issue of whether the ratings used in the AIRE instrument correlate with what would be meaningful to those assessing the quality of a dental service. For example, one recommended publication focusses heavily on the number of patients with fillings, dentures, periodontal disease etc, as a proportion of the total population sampled.³ Using the UK as an example, this raises the question of whether there is a correlation between what scores highly on the AIRE and the type of outcomes a quality NHS dental service should expect to see. Proportions of people with treatment suggestive of decay experience will correlate highly with the Index of Multiple Deprivation of any area.⁴ It would be incorrect to suggest that a 'quality' dental service can be assessed by the levels of dental disease in the population sampled. All publications included in this review describing the measures development process were either from Europe or from the United States. There is wide variation in how (oral) healthcare systems are funded and managed across these areas and this is likely to influence the applicability of certain measures in different systems.

Many of the measures overlap but have different ratings. This highlights that the review does not merely rate the measures themselves. It also rates, for example, how much the evidence is cited on these. This is useful in reminding us of the importance of clearly stating any evidence sources and describing any stakeholder engagement when publishing outcomes measures. The review also highlights the scarcity of measures for (frail) elderly populations which is an increasing area of focus in oral healthcare.

The review was not designed to provide a list of the 'best' outcome measures as these will need to be varied dependent on the specific service. It will undoubtedly be useful to anyone developing or assessing outcome measures for dental services in highlighting important factors for consideration. When assessing quality it is important to consider that 'not everything that can be counted counts, and not everything that counts can be counted.⁵ This review provides a helpful framework when considering quantitative outcome measures with defined and available numerators and denominators. The inclusion of patient-reported outcome measures shows an important progression from focusing on clinical outcome measures and access to dental care. There may in future be opportunities for developing frameworks for gathering less quantitative aspects and 'soft intelligence' to encapsulate the whole patient experience when assessing the quality of oral healthcare.

Practice points

• Stakeholder views, clinical evidence and an assessment of the reliability and validity of the measures are very important when developing quality measures for dental care

• A useful list of quality measures in current or recent use can be found in Appendix S5

• Only four of the reviewed projects piloted their measures and this is an important consideration when developing and assessing quality measures