Introduction

Despite the dramatic increase in telemedicine use in 2020, equity in telemedicine appointment access and use remains a concern. Reduced access to telemedicine may be due to the digital divide (e.g., lack of access to internet or devices, low digital literacy) as well as more general barriers influencing access to both in-person and virtual care (e.g., communication barriers, scheduling logistics, payment).1,2 These concerns were validated by early coronavirus disease 2019 (COVID-19) pandemic reports that described inequities in telemedicine use across different specialties by race and ethnicity, insurance, language, and access to broadband internet.3,4,5,6,7,8,9,10,11,12,13,14

Telemedicine use in pediatric primary care was limited prior to 2019 but increased exponentially in 2020 in the context of the COVID-19 pandemic.15,16,17,18,19 This differed from pediatric subspecialty care and pediatric mental health, where telemedicine use also increased during the pandemic, but where clinicians may have had pre-pandemic experience in telemedicine use.20 The barriers to telemedicine uptake in pediatric primary care were similar to those of primary care internists regarding equipment, technical support, and training needs,21 but pediatrics also faced the unique challenge of requiring connection between the caregiver–child–clinician triad, rather than a patient-clinician dyad. The impact of unequal access to telemedicine by socioeconomic status is especially concerning in pediatric primary care as children are disproportionately impacted by poverty as well as other health-related social needs. One in seven children (14.4%) live in poverty, which is more than any other age group in the United States (9.4% of 18–64 year olds, 8.9% of adults 65 and older).22 Children are also the most diverse age group in the United States, where most children under 5 years old are children of color,22 further heightening the need to strive for equity in our care delivery, including virtual care delivery.

Given the unique contexts for the transition to incorporating telemedicine in pediatric primary care, we examined the evolution of primary care telemedicine appointments over the course of the first year of pediatric primary care telemedicine experience (April 2020–March 2021). To identify priorities for enhanced equity in access to primary care telemedicine, we described characteristics of scheduled primary care telemedicine appointments, and factors associated with subsequent telemedicine appointment attendance informed by Levesque et al.’s patient-centered model of access to care and literature specifically on equity in telemedicine.23,24,25,26 We specifically assessed differences in the association between these factors and telemedicine appointment scheduling and attendance over twelve months as caregivers, patients, clinicians, and systems gained experience with telemedicine.

Methods

Context

Electronic health record (EHR) data was analyzed for two academic primary care pediatric practices in Allegheny County, Pennsylvania. Both practices are patient-centered medical homes providing primary care for 16,754 children per year across a total of 27,154 visits in 2019. Neither practice provided primary care telemedicine before March 2020. Primary care telemedicine appointments were offered beginning March 23, 2020.27 At this time, the clinics also reduced evening and weekend hours due to staffing limitations. Pre-pandemic, practices operated individual clinician schedules as well as one shared acute care clinic schedule staffed by a group of attendings and residents to accommodate acute, same-day, and walk-in patients. During the pandemic, most primary care telemedicine appointments were scheduled on the acute care clinic schedule, although some were scheduled with individual clinicians. Throughout the first year of pediatric primary care telemedicine experience, specific changes that occurred included gradual re-opening of in-person care options (e.g., evening in-person appointments resumed in May 2020), increased use of one specific EHR-embedded telemedicine platform requiring portal access (which became the primary means of telemedicine appointments as of June 2020), and evolving nurse triage protocols and in-clinic COVID-19 testing capabilities (throughout the study period).

Data

We extracted encounter data from the EHR for scheduled telemedicine appointments across both sites from March 2020 to March 2021. Scheduled telemedicine appointments were identified using an administrative code indicating telemedicine appointment type within the EHR. In these practices, visits were never scheduled as audio-only, but some were completed as audio-only if a video connection could not be established. Such visits could not be reliably identified in the available data, therefore all completed telemedicine visits are grouped together rather than distinguishing audio-video from audio-only. As telemedicine appointments did not begin until the final week of March, we excluded from the analysis the 190 appointments that occurred during March in order to focus on 12 complete months of data (April 2020–March 2021). We excluded appointments scheduled for preventive care via telemedicine (n = 254) from analyses, because these appointments were scheduled during a three-month preventive care initiative (April–June 2020) which was subsequently abandoned. To compare clinic patient panels with patients using telemedicine, we obtained demographic data on active patients within the two practices, defined as patients with one or more encounters at either practice within the two-year period from January 2019 to December 2020.

Child and family characteristics

Building on Levesque et al.’s patient-centered model of access to care and literature specifically on equity in telemedicine, we focused on factors potentially associated with barriers to initially scheduling and subsequently attending a telemedicine appointment.23,24,25,26 We extracted child and family characteristics including child age at time of appointment, race and ethnicity, family language preference, health insurance type (i.e., Medicaid, commercial, self-pay), date of enrollment in the EHR-embedded patient portal for each child, and ZIP code of patient’s primary address from the EHR for each child within the practice’s patient panel, including those with a scheduled telemedicine appointment. The child’s race, ethnicity and family language preference are recorded in the EHR based on the parent’s self-report during the child’s first visit with the practice. Based on patient portal enrollment date, families were categorized as “early adopters” if enrollment date was prior to March 2020 for children born prior to 2020, and for children born after March 2020, if enrollment date was within one month of birth. Using child’s ZIP code, we determined the percentages of households within each ZIP code that had no access to internet or computers and those without access to a private vehicle based on the 2019 American Community Survey 5-year estimates.28 Fourteen individuals with missing or unmatched ZIP code data were also excluded from the final cohort.

Appointment characteristics

For each scheduled telemedicine appointment, we obtained appointment characteristics including practice site, date of appointment, date of appointment scheduling, appointment reason given at time of scheduling, intended telemedicine platform and appointment outcome (i.e., attended, not attended). We categorized the stated reason for appointment at time of scheduling into one of 10 appointment reasons (i.e., COVID-19; dermatologic; fever; follow-up; gastrointestinal/genitourinary [GI/GU]; head, eyes, ears, nose, throat [HEENT]; mental health; musculoskeletal/neurologic; other/unknown; respiratory) and categorized appointments scheduled the same calendar day as “same-day.” We extracted the intended telemedicine platform from the appointment notes made at the time of scheduling and categorized as either EHR-embedded or non-EHR-embedded platform. We categorized an appointment as “attended” if it was attended by both patient and clinician and “not attended” if the patient did not attend the appointment, canceled, or left without being seen by a clinician.

Analysis

We describe characteristics of the entire population of active patients in the practices and of children with a scheduled telemedicine appointment from April 2020 to March 2021. To examine differences in the proportion of scheduled telemedicine appointments for each child-, family-, and appointment-level characteristic over time, we stratified scheduled appointments into four quarters (April–June 2020, July–September 2020, October–December 2020, and January–March 2021) and compared characteristics across quarters using Pearson χ2 tests.

We report modeled percentages of attended primary care telemedicine appointments and 95% confidence intervals (95% CIs) adjusted for child-, family-, and appointment-level characteristics. To examine adjusted percentages of attended telemedicine appointments by child-, family-, and appointment-level characteristics, we conducted logistic regression and mixed effects logistic regression to account for clustering of visits within individual children. Independent variables for both models included child- (i.e., age, race and ethnicity, insurance), family- (i.e., family language preference, patient portal adoption, ZIP code household computer access, ZIP code household access to a private vehicle), appointment-level factors (i.e., appointment reason, same-day appointment, clinician schedule type, and intended telemedicine platform), and calendar month. The mixed effect logistic regression model did not significantly improve the fit over the logistic regression model, so we present the simpler logistic regression model in our main results. Finally, interactions between calendar month and each statistically significant (alpha < 0.05) independent variable in the initial logistic regression model were tested and retained if statistically significant. All analyses were conducted in Stata version 16.1 (StataCorp) with significance assessed using a 2-sided alpha level of 0.05.

Results

Between April 2020 and March 2021, there were 5178 scheduled telemedicine appointments for 3254 children. In comparison to the practices’ active patients, children with scheduled telemedicine appointments appeared to be younger (mean age 7.4 vs. 8.5 years) and more likely to be patient portal early adopters (23% vs. 7%), but with similar distribution of parent-reported race and ethnicity and health insurance (Table 1).

Table 1 Characteristics of entire population cared for and children with a primary care telemedicine appointment scheduled between April 2020 and March 2021 within two pediatric primary care practices.
Full size table

Characteristics of scheduled primary care telemedicine appointments

The proportion of scheduled telemedicine appointments by quarter did not differ by child age, race and ethnicity, insurance type, early patient portal adopter status, or neighborhood private vehicle access (Table 2). The proportion of appointments scheduled for children with a family preferred language other than English or Spanish was higher in quarter 4 (6%) than in quarter 1 (4%, p = 0.01). The proportion of appointments scheduled for children residing in ZIP codes with the lowest household computer and internet access differed over time, from 24% in quarter 1 to 19% in quarter 3 (p = 0.01). The proportion of telemedicine appointments scheduled for specific appointment reasons also differed by quarter (Table 3). We also observed a difference by quarter in the proportion of appointments scheduled as same-day appointments (67% in quarter 1 vs 76% in quarter 4, p < 0.001) and with the intention to attend via the EHR-embedded telemedicine platform (47% in quarter 1 vs. 95% in quarter 2, p < 0.001).

Table 2 Characteristics of scheduled telemedicine appointments by quarter, April 2020–March 2021.
Full size table
Table 3 Appointment characteristics of scheduled primary care telemedicine appointments (2020–2021).
Full size table

Attendance at primary care telemedicine appointments

Overall, 80% of scheduled primary care telemedicine appointments scheduled between April 2020-March 2021 were attended. Likelihood of attending a scheduled telemedicine appointment was highest for children who identified as white non-Hispanic (85%, 95% CI 82–87%), had commercial insurance (85%, 95% CI 82–87%), families who preferred speaking a language other than English or Spanish (90%, 95% CI 86–94%), were early patient portal adopters (83%, 95% CI 81–85%), had COVID-19 as appointment reason (85%, 95% CI 83–88%), scheduled a same-day appointment (86%, 95% CI 85–87%), and intended to attend via non-EHR-embedded telemedicine platform (83%, 95% CI 81–85%, Table 4).

Table 4 Relative risk and modeled percent of children who attended primary care telemedicine appointments by child, family and appointment characteristics between April 2020 and March 2021a.
Full size table

There were significant interaction effects between month of scheduled appointment and family preferred language (p = 0.03) and appointment reason (p = 0.03). Families preferring Spanish language had higher attendance in later months compared to earlier months, while attendance for families preferring a non-Spanish language did not differ over time. The appointment reason of “follow-up” was less likely to be attended than any other visit reason over time. Additional interaction terms were not statistically significant.

Discussion

Among children who receive primary care in two academic primary care pediatric practice sites, we found scheduling and attendance of primary care telemedicine appointments differed significantly by child-, family-, and appointment-level characteristics. These findings add to the existing literature illustrating the need for careful attention to equity in pediatric primary care telemedicine use.

Some characteristics associated with changes in scheduling by quarter suggest evolution in clinical needs of pediatric patients or in clinical uses of telemedicine in pediatric primary care. For example, we observed higher numbers of telemedicine appointments with COVID-19 as the primary concern in later quarters. Shifting testing capabilities within primary care offices and the community, evolving rates of COVID-19 infection among children in our community, and solidification of processes to evaluate and coordinate testing for patients through telemedicine likely contributed to this observation.29,30,31,32 Higher rates of same-day scheduling over time suggests an evolving use of telemedicine focused increasingly on acute illnesses, potentially reinforced by the lowest attendance occurring for appointments scheduled for “follow-up” (primarily indicating primary care visits scheduled after emergency department or hospital visits). Perhaps relatedly, we were surprised to find that only 5% of telemedicine appointments were scheduled for mental health concerns by later 2021, despite significant attention to mental health needs at this point during the pandemic. While on the lower side this proportion is within the range of primary care telemedicine visits dedicated to mental health in prior studies (0.4–30%).16,33,34,35,36,37 Potential contributors to this wide range of mental health telemedicine use include challenges with the ability to guarantee privacy, or perceived necessity of in-person measurements (e.g., blood pressure for stimulant medication monitoring) resulting in differential telemedicine adoption and use for mental health visits across primary care settings.

Some of the differences in scheduled telemedicine appointments may reflect externally driven changes over time. For example, an institutional policy prepared for the eventual end of the Public Health Emergency by mandating use of the EHR-embedded telemedicine platform whenever possible by summer 2020, to ensure compliance with the health insurance portability and accountability act (HIPAA).38 Prior to this change, clinicians used many platforms based on family and technology capabilities and needs. Once limited to one specific platform, clinicians could no longer tailor platform use based on patient needs, limiting access for families that may have benefited from texting links to connect, bringing a third party into a visit, connecting without downloading a specific application, or launching a visit without a legal guardian with a recognized proxy relationship present. Despite the substantial implications of these changes, we did not observe changes in attendance rate over time overall or among child or family characteristics other than family language. However, we did observe shifts in scheduling for households in ZIP codes with lower technology access, suggesting patients and practices may have scheduled differently in later quarters due to anticipated connectivity barriers.

While the percentage of scheduled telemedicine appointments over time did not vary by the child’s race and ethnicity or insurance type, these characteristics were associated with overall attendance. This finding is consistent with concerns about equity in access to telemedicine. Even among families who are interested in telemedicine and who successfully schedule an appointment, there is still differential ability to attend the appointment likely due to far-reaching and interrelated impacts of systemic racism, socioeconomic barriers, or technical constraints.1 The inequity in attendance by families who prefer Spanish language improved over time which coincided with staff helping Spanish-speaking families to enroll in the patient portal, developing written information in Spanish on how to complete a telemedicine appointment, and the growing ability to integrate interpreters into visits. In contrast, the disparities in appointment completion rate by race and ethnicity and insurance did not change over time, indicating this neither worsened nor improved, such that active efforts on the system level will be needed to reduce these differences. Multi-faceted systemic interventions have been proposed including infrastructure interventions to improve access to broadband internet and internet-capable devices at the federal level, telehealth platform interventions at the institutional level to ensure mobile health tools are designed for and with patients with potential barriers (e.g., non-English language, digital literacy) using human-centered design.26

Our analysis has key limitations including our focus on two primary care pediatric sites which care for a population of children who are publicly insured and predominantly identify as Black, and thus may not generalize to practices with other patient composition. However, academic pediatric primary care sites are responsible for training large numbers of future pediatricians and therefore the experience with telemedicine in academic spaces may have influence beyond the sites themselves. We also note that the COVID-19 pandemic, and telemedicine’s role in healthcare, is constantly evolving. Thus, it is important to interpret our findings in their temporal context and to continue assessing telemedicine use in pediatric primary care. As this analysis was retrospective and observational in nature, we are limited to data available in the EHR, and therefore we cannot determine events upstream of appointment scheduling (e.g., contact with office, appointment offered, appointment declined). Finally, we note this is a retrospective analysis of EHR data, which lacks data on technology and internet access on the individual level. We used 5-year estimates from the ACS to approximate these variables on a community level as the sample size in each zip code was too small (i.e., <65,000) to use the 1-year ACS estimates.

In early 2020, there was some thought that telemedicine was a fad – that it was a good solution for the needs at the time but that utility would wane. While the volume of telemedicine appointments did decline over the course of the first year in use, use persisted well above pre-pandemic rates. Among those who scheduled a telemedicine appointment, attendance remained approximately 80% throughout the study which also supports sustainability.39,40 This analysis also signals that same-day scheduling of appointments may be where patients and clinicians find the most utility within primary care (indicated by higher proportion of appointments scheduled same-day in later quarters with sustained completion rate of the visits). We observed a similar distribution by quarter of scheduled telemedicine appointments across child race and ethnicity, insurance type, computer or internet access, and access to a private vehicle, indicating that for those who have historically encountered barriers to accessing traditional brick and mortar healthcare settings, telemedicine was a viable way to deliver healthcare beyond the initial months of the pandemic.41,42,43,44 Also, while visit attendance did not change over time, the proportion of appointments scheduled consisted of more families who preferred a non-English language, suggesting improvement in language equity. Taken together, this illustrates the potential to slowly move towards more equitable scheduling and completion of telemedicine appointments within pediatric primary care, but also highlight ongoing need for further improvement.

Conclusion

Among children who receive primary care in two academic pediatric primary care practice sites, we found scheduling and attendance of primary care telemedicine appointments varied significantly with child-, family-, and appointment-level characteristics. These factors evolved during the first year of telemedicine experience (April 2020–March 2021). Among those who chose to schedule a telemedicine appointment, approximately four in five attended the appointment. We found disparities in both scheduling and attending telemedicine appointments, but signs of greater language equity over time. Specifically, we observed higher percentages of appointments scheduled by non-English speaking families compared to English-speaking families and narrowing of differential appointment completion rates by family-preferred language. These findings illustrate the need for careful attention to equity in telemedicine use to ensure that telehealth functions as a tool to support rather than undermine equity in pediatric primary care.