Sneha Chotaliya

We are often surrounded by an abundance of research and articles, but the quality and validity can vary massively. Not everything will be of a good quality - or even valid. An important part of reading a paper is first assessing the paper. This is a key skill for all healthcare professionals as anything we read can impact or influence our practice. It is also important to stay up to date with the latest research and findings.

Whilst critical appraisal can be a whole course for students, this article aims to provide an introduction to help students to get started with the right questions, and make critical appraisal seem less daunting. Here are some of the tools and basic considerations you might find useful when critically appraising an article.

In a nutshell when appraising an article, you are assessing:1

  • Its relevance, methods, and validity

  • The strengths and weaknesses of the paper

  • Relevance to specific circumstances.2

In this article, we aim to highlight the key parts of critical appraisal and mention some potential pitfalls of appraisal.

By assessing the strengths and weaknesses of the whole paper, we can start to understand the potential impact of this on the results

Appraisal is not solely assessing the results or methods as standalone sections. It involves assessment of the whole paper. By assessing the strengths and weaknesses of the whole paper, we can start to understand the potential impact of this on the results.

Using Critical Appraisal Frameworks

Frameworks provide a holistic, logical, and stepwise approach to assessing articles. It covers all the key areas of appraisal and provides useful prompts for further analysis. Using the correct framework for the study design is essential to ask relevant questions.

Most appraisal structures focus on the research question, methods, results, and applicability of the conclusion. Different frameworks (suited to different study designs) allow us to consider if the method adopted in the study is scientifically valid through many criteria. For example, has bias been introduced in selection, data collection, statistical analysis, or data presentation? Any conflicting interests should be compared with the aims and objectives of the paper.

After considering the method, we must consider the results of the study. The conclusion drawn from these results should be valid and appropriate. A thorough assessment should be conducted for any results missing from analysis, the statistical method used and if this was appropriate and used correctly. Are the results significant? Or are they uncertain?

Other appraisal tools:

  • Qualitative: Robert Wood Johnson Foundation

  • Systematic Reviews: Joanna Briggs Institute

  • The Centre for Evidence Based Medicine at University of Oxford

  • Evidence Based Medicine: 'How to Read a Paper: The Basics of Evidence-based Medicine and Healthcare' by Professor Trisha Greenhalgh

  • EQUATOR Network, Enhancing the QUAlity and Transparency Of health Research.

The Critical Appraisal Skills Programme (CASP) tool

CASP has specific checklists to use for critically appraising randomised controlled trials, systematic reviews, qualitative studies, cohort studies, diagnostic studies and case control studies.3

  • The research question should be concise and focused. This should be clearly highlighted in the title and abstract. In particular, the relevant population, intervention/exposure/risk factor, comparisons/controls and the outcome of interest should be stated well.

  • This question will enable you to understand whether the correct and most appropriate study design has been chosen to answer the question intended:4

    • Randomised controlled trials: patients are randomly allocated to experimentally receive or not receive an intervention. This is the best study design to assess the effectiveness of a treatment.

    • Cohort study: follows up a particular cohort of people of interest who may have an exposure to see whether they obtain the designated outcome. This group's outcome would be compared with a control group without the exposure. This study allows observance of temporal associations.

    • Cross-sectional study: collects data at one point in time to give understanding about a particular feature of the intended population.

      These should have been considered and any relevant adjustments considered to minimise its effect

Population of interest

  • The inclusion and exclusion criteria should be clearly stated. Any specific details should be included.4

  • Any measures taken to reduce bias will be found in the methods section.

  • Any conclusion may only be valid for the population studied. But have any groups of the studied population been excluded? This would be an example of selection bias.

Methods

  • Details about the intervention or outcome data and how they were collected, and collection accuracy should be carefully analysed. Information about how the endpoint was measured and categorised should have been included.

  • Confounding factors can introduce bias in any study, especially observational studies. These are factors that are independently associated with both the exposure and outcome variables. These should have been considered and any relevant adjustments considered to minimise their effect. Depending on the study question, confounding data may have been collected, so it is worth bearing this in mind throughout your appraisal.4

  • Details of data not collected or presented should be provided with an appropriate explanation.

Results

  • The authors should give information about any missing or omitted data with a reasonable justification.

  • Similarly, one must think about which populations the results are applicable to. It is useful to see how the results of this one study compare with other studies investigating a similar question. A meta-analysis can provide a statistical insight into this.

Conclusion

It is critically important to analyse, inspect and assess every scientific paper or journal we may read before believing or implementing it. A thorough and rigorous approach means that every study and resulting paper is held accountable to its claims. As scientific research and the concept of evidence-based practice in dentistry expands, so does the responsibility of peer-review and critical appraisal. This will be a key and critical skill for every healthcare professional and student to possess.