Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Selection bias in rheumatic disease research

Key Points

  • Unlike research findings on risk factors for incident conditions, the evidence on risk factors for disease sequelae among patients with rheumatic diseases have often been inconsistent or paradoxical

  • Although biological explanations for these counterintuitive results might exist, an enticing methodological explanation is a type of selection bias called index event bias, which can affect research on disease sequelae

  • Propensity score methods or active comparator analysis in pharmaco-epidemiological research helps to address confounding issues in observational studies, but does not address selection bias owing to potential differential loss to follow-up

  • The depletion of susceptibles can explain the decreasing impact of risk factors on mortality with ageing in rheumatic conditions, as well as explain the null (or inverse) associations of prevalent exposure studies

  • To avoid these issues, investigators should carefully specify the research question of interest and clarify the time sequence of exposures, mediators, and outcome variables

  • Furthermore, investigators should use incident exposures whenever possible, minimize loss to follow-up, and exercise proper inference

Abstract

The identification of modifiable risk factors for the development of rheumatic conditions and their sequelae is crucial for reducing the substantial worldwide burden of these diseases. However, the validity of such research can be threatened by sources of bias, including confounding, measurement and selection biases. In this Review, we discuss potentially major issues of selection bias—a type of bias frequently overshadowed by other bias and feasibility issues, despite being equally or more problematic—in key areas of rheumatic disease research. We present index event bias (a type of selection bias) as one of the potentially unifying reasons behind some unexpected findings, such as the 'risk factor paradox'—a phenomenon exemplified by the discrepant effects of certain risk factors on the development versus the progression of osteoarthritis (OA) or rheumatoid arthritis (RA). We also discuss potential selection biases owing to differential loss to follow-up in RA and OA research, as well as those due to the depletion of susceptibles (prevalent user bias) and immortal time bias. The lesson remains that selection bias can be ubiquitous and, therefore, has the potential to lead the field astray. Thus, we conclude with suggestions to help investigators avoid such issues and limit the impact on future rheumatology research.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: A causal diagram illustration of index event bias, also known as collider stratification bias.
Figure 2: A causal diagram of a typical observational study showing the assessment of the effect of obesity on OA progression among patients with (incident) OA.
Figure 3: A causal diagram of a typical observational study showing the assessment of the effect of smoking on RA progression (or CVD complications) among patients with RA.
Figure 4: Causal diagrams displaying the effect of smoking on CVD.
Figure 5: Differential loss to follow-up in studies of RA therapy.
Figure 6: Immortal time bias as a form of selection bias.

Similar content being viewed by others

References

  1. Reginster, J. Y. The prevalence and burden of arthritis. Rheumatology (Oxford) 41 (Suppl. 1), 3–6 (2002).

    Article  Google Scholar 

  2. Symmons, D. P. & Gabriel, S. E. Epidemiology of CVD in rheumatic disease, with a focus on RA and SLE. Nat. Rev. Rheumatol. 7, 399–408 (2011).

    Article  Google Scholar 

  3. Gabriel, S. E. Heart disease and rheumatoid arthritis: understanding the risks. Ann. Rheum. Dis. 69 (Suppl. 1), i61–i64 (2010).

    Article  Google Scholar 

  4. Eder, L. et al. The association between smoking and the development of psoriatic arthritis among psoriasis patients. Ann. Rheum. Dis. 71, 219–224 (2012).

    Article  Google Scholar 

  5. Zhang, Y. et al. Methodologic challenges in studying risk factors for progression of knee osteoarthritis. Arthritis Care Res. (Hoboken) 62, 1527–1532 (2010).

    Article  Google Scholar 

  6. Canto, J. G. et al. Number of coronary heart disease risk factors and mortality in patients with first myocardial infarction. JAMA 306, 2120–2127 (2011).

    Article  CAS  Google Scholar 

  7. Zhang, Y. & Jordan, J. M. Epidemiology of osteoarthritis. Rheum. Dis. Clin. North Am. 34, 515–529 (2008).

    Article  Google Scholar 

  8. Felson, D. T. et al. Osteoarthritis: new insights. Part 1: the disease and its risk factors. Ann. Intern. Med. 133, 635–646 (2000).

    Article  CAS  Google Scholar 

  9. Belo, J. N., Berger, M. Y., Reijman, M., Koes, B. W. & Bierma-Zeinstra, S. M. Prognostic factors of progression of osteoarthritis of the knee: a systematic review of observational studies. Arthritis Rheum. 57, 13–26 (2007).

    Article  CAS  Google Scholar 

  10. Zhang, Y. et al. Bone mineral density and risk of incident and progressive radiographic knee osteoarthritis in women: the Framingham Study. J. Rheumatol 27, 1032–1037 (2000).

    CAS  PubMed  Google Scholar 

  11. Hart, D. J. et al. The relationship of bone density and fracture to incident and progressive radiographic osteoarthritis of the knee: the Chingford Study. Arthritis Rheum. 46, 92–99 (2002).

    Article  Google Scholar 

  12. Lane, N. E. et al. Wnt signaling antagonists are potential prognostic biomarkers for the progression of radiographic hip osteoarthritis in elderly Caucasian women. Arthritis Rheum. 56, 3319–3325 (2007).

    Article  CAS  Google Scholar 

  13. McAlindon, T. E. et al. Do antioxidant micronutrients protect against the development and progression of knee osteoarthritis? Arthritis Rheum. 39, 648–656 (1996).

    Article  CAS  Google Scholar 

  14. Vesperini, V. et al. Tobacco exposure reduces radiographic progression in early rheumatoid arthritis. Results from the ESPOIR cohort. Arthritis Care Res. (Hoboken) 65, 1899–1906 (2013).

    Article  CAS  Google Scholar 

  15. Harrison, B. J., Silman, A. J., Wiles, N. J., Scott, D. G. & Symmons, D. P. The association of cigarette smoking with disease outcome in patients with early inflammatory polyarthritis. Arthritis Rheum. 44, 323–330 (2001).

    Article  CAS  Google Scholar 

  16. Finckh, A., Dehler, S., Costenbader, K. H. & Gabay, C. Cigarette smoking and radiographic progression in rheumatoid arthritis. Ann. Rheum. Dis. 66, 1066–1071 (2007).

    Article  CAS  Google Scholar 

  17. Gonzalez, A. et al. Do cardiovascular risk factors confer the same risk for cardiovascular outcomes in rheumatoid arthritis patients as in non-rheumatoid arthritis patients? Ann. Rheum. Dis. 67, 64–69 (2008).

    Article  CAS  Google Scholar 

  18. Naranjo, A. et al. Cardiovascular disease in patients with rheumatoid arthritis: results from the QUEST-RA study. Arthritis Res. Ther. 10, R30 (2008).

    Article  Google Scholar 

  19. Manson, J. E. et al. A prospective study of obesity and risk of coronary heart disease in women. N. Engl. J. Med. 322, 882–889 (1990).

    Article  CAS  Google Scholar 

  20. Escalante, A., Haas, R. W. & del Rincon, I. Paradoxical effect of body mass index on survival in rheumatoid arthritis: role of comorbidity and systemic inflammation. Arch. Intern. Med. 165, 1624–1629 (2005).

    Article  Google Scholar 

  21. Wilson, P. W. et al. Prediction of coronary heart disease using risk factor categories. Circulation 97, 1837–1847 (1998).

    Article  CAS  Google Scholar 

  22. Myasoedova, E. et al. Lipid paradox in rheumatoid arthritis: the impact of serum lipid measures and systemic inflammation on the risk of cardiovascular disease. Ann. Rheum. Dis. 70, 482–487 (2011).

    Article  CAS  Google Scholar 

  23. Peters, M. J. et al. EULAR evidence-based recommendations for cardiovascular risk management in patients with rheumatoid arthritis and other forms of inflammatory arthritis. Ann. Rheum. Dis. 69, 325–331 (2009).

    Article  Google Scholar 

  24. Solomon, D. H., Peters, M. J., Nurmohamed, M. T. & Dixon, W. Unresolved questions in rheumatology: motion for debate: the data support evidence-based management recommendations for cardiovascular disease in rheumatoid arthritis. Arthritis Rheum. 65, 1675–1683 (2013).

    Article  Google Scholar 

  25. Li, W., Han, J. & Qureshi, A. A. Smoking and risk of incident psoriatic arthritis in US women. Ann. Rheum. Dis. 71, 804–808 (2011).

    Article  Google Scholar 

  26. Bowcock, A. M. & Cookson, W. O. The genetics of psoriasis, psoriatic arthritis and atopic dermatitis. Hum. Mol. Genet. 13 (Suppl. 1), R43–R55 (2004).

    Article  CAS  Google Scholar 

  27. Duffin, K. C. et al. Genetics of psoriasis and psoriatic arthritis: update and future direction. J. Rheumatol 35, 1449–1453 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Aune, E., Roislien, J., Mathisen, M., Thelle, D. S. & Otterstad, J. E. The “smoker's paradox” in patients with acute coronary syndrome: a systematic review. BMC Med. 9, 97 (2011).

    Article  Google Scholar 

  29. Romero-Corral, A. et al. Association of bodyweight with total mortality and with cardiovascular events in coronary artery disease: a systematic review of cohort studies. Lancet 368, 666–678 (2006).

    Article  Google Scholar 

  30. Lavie, C. J., De Schutter, A., Patel, D., Artham, S. M. & Milani, R. V. Body composition and coronary heart disease mortality—an obesity or a lean paradox? Mayo Clin. Proc. 86, 857–864 (2011).

    Article  CAS  Google Scholar 

  31. Dahabreh, I. J. & Kent, D. M. Index event bias as an explanation for the paradoxes of recurrence risk research. JAMA 305, 822–823 (2011).

    Article  CAS  Google Scholar 

  32. Kent, D. M. & Thaler, D. E. Is patent foramen ovale a modifiable risk factor for stroke recurrence? Stroke 41, S26–S30 (2010).

    Article  Google Scholar 

  33. Tyas, S. L. et al. Transitions to mild cognitive impairments, dementia, and death: findings from the Nun Study. Am. J. Epidemiol. 165, 1231–1238 (2007).

    Article  Google Scholar 

  34. Glymour, M. M. Invited commentary: when bad genes look good—APOE*E4, cognitive decline, and diagnostic thresholds. Am. J. Epidemiol. 165, 1239–1246; author reply 1247 (2007).

    Article  Google Scholar 

  35. Baglin, T. Unraveling the thrombophilia paradox: from hypercoagulability to the prothrombotic state. J. Thromb. Haemost. 8, 228–233 (2010).

    Article  CAS  Google Scholar 

  36. Hernandez-Diaz, S., Schisterman, E. F. & Hernan, M. A. The birth weight “paradox” uncovered? Am. J. Epidemiol. 164, 1115–1120 (2006).

    Article  Google Scholar 

  37. Myers, J. et al. The obesity paradox and weight loss. Am. J. Med. 124, 924–930 (2011).

    Article  Google Scholar 

  38. VanderWeele, T. J., Mumford, S. L. & Schisterman, E. F. Conditioning on intermediates in perinatal epidemiology. Epidemiology 23, 1–9 (2011).

    Article  Google Scholar 

  39. VanderWeele, T. J. & Robins, J. M. Directed acyclic graphs, sufficient causes, and the properties of conditioning on a common effect. Am. J. Epidemiol. 166, 1096–1104 (2007).

    Article  Google Scholar 

  40. Smits, L. J. et al. Index event bias—a numerical example. J. Clin. Epidemiol. 66, 192–196 (2013).

    Article  Google Scholar 

  41. Westreich, D. & Greenland, S. The Table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am. J. Epidemiol. 177, 292–298 (2013).

    Article  Google Scholar 

  42. Valeri, L. & Vanderweele, T. J. Mediation analysis allowing for exposure-mediator interactions and causal interpretation: theoretical assumptions and implementation with SAS and SPSS macros. Psychol. Methods 18, 137–150 (2013).

    Article  Google Scholar 

  43. Zhang, Y. et al. What effect is really being measured? An alternative explanation of paradoxical phenomenon in studies of osteoarthritis progression. Arthritis Care & Res (Hoboken) http://dx.doi.org/10.1002/acr.22213.

  44. Felson, D. T. et al. Risk factors for incident radiographic knee osteoarthritis in the elderly: the Framingham Study. Arthritis Rheum. 40, 728–733 (1997).

    Article  CAS  Google Scholar 

  45. Cooper, C. et al. Risk factors for the incidence and progression of radiographic knee osteoarthritis. Arthritis Rheum. 43, 995–1000 (2000).

    Article  CAS  Google Scholar 

  46. Grijalva, C. G. et al. Initiation of tumor necrosis factor-alpha antagonists and the risk of hospitalization for infection in patients with autoimmune diseases. JAMA 306, 2331–2339 (2011).

    Article  CAS  Google Scholar 

  47. Solomon, D. H. et al. Association between disease-modifying antirheumatic drugs and diabetes risk in patients with rheumatoid arthritis and psoriasis. JAMA 305, 2525–2531 (2011).

    Article  CAS  Google Scholar 

  48. O'Dell, J. R. et al. Therapies for active rheumatoid arthritis after methotrexate failure. N. Engl. J. Med. 369, 307–318 (2013).

    Article  CAS  Google Scholar 

  49. O'Dell, J. R. et al. Treatment of rheumatoid arthritis with methotrexate and hydroxychloroquine, methotrexate and sulfasalazine, or a combination of the three medications: results of a two-year, randomized, double-blind, placebo-controlled trial. Arthritis Rheum. 46, 1164–1170 (2002).

    Article  CAS  Google Scholar 

  50. O'Dell, J. R. et al. Treatment of rheumatoid arthritis with methotrexate alone, sulfasalazine and hydroxychloroquine, or a combination of all three medications. N. Engl. J. Med. 334, 1287–1291 (1996).

    Article  CAS  Google Scholar 

  51. Dixon, W. & Felson, D. T. Is anti-TNF therapy safer than previously thought? JAMA 306, 2380–2381 (2011).

    Article  CAS  Google Scholar 

  52. Hernan, M. A., Hernandez-Diaz, S. & Robins, J. M. Randomized trials analyzed as observational studies. Ann. Intern. Med. http://dx.doi.org/10.7326/0003-4819-159-8-201310150-00709.

  53. Bongartz, T. et al. Anti-TNF antibody therapy in rheumatoid arthritis and the risk of serious infections and malignancies: systematic review and meta-analysis of rare harmful effects in randomized controlled trials. JAMA 295, 2275–2285 (2006).

    Article  CAS  Google Scholar 

  54. Little, R. J. et al. The prevention and treatment of missing data in clinical trials. N. Engl. J. Med. 367, 1355–1360 (2012).

    Article  CAS  Google Scholar 

  55. Doll, R. & Hill, A. B. Mortality of British doctors in relation to smoking: observations on coronary thrombosis. Natl Cancer Inst. Monogr. 19, 205–268 (1966).

    CAS  PubMed  Google Scholar 

  56. Wolfe, F. & Michaud, K. Effect of body mass index on mortality and clinical status in rheumatoid arthritis. Arthritis Care Res. (Hoboken) 64, 1471–1479 (2012).

    Article  Google Scholar 

  57. Nguyen, U. S., Niu, J., Choi, H. K. & Zhang, Y. Body mass index and mortality: comment on article by Wolfe and Michaud. Arthritis Care Res. (Hoboken) 65, 834–835 (2013).

    Article  Google Scholar 

  58. Choi, H. K. et al. The risk of pulmonary embolism and deep vein thrombosis in rheumatoid arthritis: a UK population-based outpatient cohort study. Ann. Rheum. Dis. 72, 1182–1187 (2013).

    Article  Google Scholar 

  59. Zoller, B., Li, X., Sundquist, J. & Sundquist, K. Risk of pulmonary embolism in patients with autoimmune disorders: a nationwide follow-up study from Sweden. Lancet 379, 244–249 (2012).

    Article  Google Scholar 

  60. Grodstein, F. & Stampfer, M. The epidemiology of coronary heart disease and estrogen replacement in postmenopausal women. Prog. Cardiovasc. Dis. 38, 199–210 (1995).

    Article  CAS  Google Scholar 

  61. Grady, D. et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann. Intern. Med. 117, 1016–1037 (1992).

    Article  CAS  Google Scholar 

  62. Manson, J. E. et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349, 523–534 (2003).

    Article  CAS  Google Scholar 

  63. Hernan, M. A. et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 19, 766–779 (2008).

    Article  Google Scholar 

  64. Levesque, L. E., Hanley, J. A., Kezouh, A. & Suissa, S. Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes. BMJ 340, b5087 (2010).

    Article  Google Scholar 

  65. Tsai, C. L. & Camargo, C. A. Jr. Methodological considerations, such as directed acyclic graphs, for studying “acute on chronic” disease epidemiology: chronic obstructive pulmonary disease example. J. Clin. Epidemiol. 62, 982–990 (2009).

    Article  Google Scholar 

  66. Rich, J. D. et al. Prior aspirin use and outcomes in acute coronary syndromes. J. Am. Coll. Cardiol. 56, 1376–1385 (2010).

    Article  Google Scholar 

Download references

Acknowledgements

This work was partly supported by grants from the NIH (NIAMS): grants R01-AR056291, R01-AR065944, K01AR064351 and P60AR047785.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed equally to researching the data for the article, discussions of the content, writing the article and editing of the manuscript before submission.

Corresponding author

Correspondence to Hyon K. Choi.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary information

Let us consider an initial cohort of 30,000 participants without rheumatoid arthritis (RA) at baseline and assume that there are four risk factors involved in the aetiology of RA incidence (E1) or progression (E2), namely the risk factor of interest (R) and three other unmeasured risk factors (U1, U2, and U3). (DOC 92 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Cite this article

Choi, H., Nguyen, US., Niu, J. et al. Selection bias in rheumatic disease research. Nat Rev Rheumatol 10, 403–412 (2014). https://doi.org/10.1038/nrrheum.2014.36

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrrheum.2014.36

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing