Introduction

Multiple pediatric subspecialties have struggled with direct patient to provider access. According to the American Academy of Pediatrics, more pediatric subspecialists are working at academic centers located in urban areas, are spending less time face to face with patients, and for some specialties such as endocrinology and neurology, appointment wait times exceed 2 weeks.1 In addition, some specialties, such as pediatric rheumatology, do not have practicing providers in every state.2 A recent study among families of children and youth with special health care needs, policy experts, and researchers indicated that further research in clinical-model refinement, specifically telemedicine, is of particularly high priority.3 As early as 1978, pediatric providers have leveraged the use of telehealth to increase access to tertiary care sites and providers4 both as a means to extend outpatient5,6 and acute inpatient specialty care.7,8 Multiple models of telemedicine care have been described in specialties such as pediatric cardiology,9,10 endocrinology,11,12 orthopedic surgery,13 and palliative care14 among others. Studies on the use of telemedicine in pediatric outpatient care in the United States have revealed potential cost-savings to families,15 increased health-related quality of life and satisfaction,16 and less school and work time missed.17 In spite of the steady growth of pediatric telemedicine use in the United States, barriers to increased adoption of telemedicine services had previously included licensing issues, paucity of provider interest, and lack of resources to train providers. For sites that had already incorporated telemedicine practices, reimbursement had remained a challenge and further limited expansion of routine telemedicine practices.18 Even for those programs with early telemedicine adoption and policies, telemedicine utilization among patients in pediatric subspecialties has been low.19

The COVID-19 pandemic, in many ways, removed these barriers to the implementation of telemedicine resulting in a rapid change in delivery of care across all sectors of health care in response to public health mandates to limit exposures of patients and health care workers to the COVID-19 infection.20 Rapid deployment of telemedicine has occurred in academic health systems and hospital networks21 leading to incredibly fast implementation across health systems as a substitute for in-person visits. At Stanford Children’s Hospital, a large tertiary care site, the number of pediatric telemedicine visits increased almost 35-fold from over a year ago; each specialty in this study had five or fewer telemedicine visits in April 2019 compared to up to 753 telemedicine visits in April 2020 (N. Pageler and T. Lee, Stanford Children’s Health, 2020). The rapid increase in pediatric subspecialty telemedicine visits has revealed potential benefits and challenges in managing children with chronic disease via telemedicine visits.22 In many ways, the COVID-19 pandemic has resulted in a natural experiment in that there are adequate volumes of telemedicine23 use across multiple pediatric specialties in order to evaluate telemedicine performance when institutional barriers are removed. We took this opportunity to evaluate the nuances of telemedicine with respect to caregiver–patient–provider communication, patient and provider experience, and clinical approaches to video visits in four different pediatric subspecialties during the rapid deployment of telemedicine at an academic children’s hospital.

Methods

Study population

This study is a cross-sectional mixed-methods study of patients, families, and providers participating in telemedicine pediatric subspecialty follow-up visits between March 23, 2020 and May 05, 2020 during the COVID-19 pandemic and was approved by the Stanford University Institutional Review Board. This study was conducted at a large, single academic children’s hospital and satellite clinics and included four subspecialties: pediatric endocrinology, pediatric nephrology, pediatric orthopedic surgery, and pediatric rheumatology. These specialties were selected as they may present unique needs such as postsurgical care, the need for specific types of bloodwork, a specific clinical exam, and varied medication monitoring needs. A convenience sample of ten patients and their caregivers (if patients were <18 years of age) from each specialty were recruited prior to their regularly scheduled video visit appointment with their specialty provider (n = 40). Each patient or patient/caregiver set in the study participated in a single telemedicine visit during the course of this study, although specialty providers may have participated in more than one observed telemedicine visit. The telemedicine visits were conducted via one of three platforms based on provider preference: via the Extended Care telehealth platform (Cisco Systems, San Jose, CA) integrated with the EHR (Epic Systems, Verona, WI), via Webex video communications platform (Cisco Systems, San Jose, CA), or via standard Zoom video communications platform (Zoom Video Communications, San Jose, CA). Neither patients/caregivers nor providers had additional training regarding telemedicine visits although they may have received specific instructions on connecting to visits and/or troubleshooting connectivity issues. These workflows varied by department. Exclusion criteria included new patient referrals and consultations, subjects under the age of 5 years, non-English-speaking subjects or non-English-speaking guardians, subjects who canceled or no-showed their scheduled appointments following consent into the study, and subjects who were unable to connect to the video visit platform and declined such visits prior to the consenting process. New referral type and young patient age were specifically excluded as triage practices for these patients varied by specialty.

Observational data

A total of 40 video visits were observed, ten per specialty. The observer was able to either (1) log in remotely to the visit if the visit used a multiparty telemedicine platform or (2) be in the clinic room with the provider and the patient/family on video. This was dependent on multiparty function capabilities of the platform or last-minute changes to the telemedicine platform. All patients and caregivers were aware that the observer was present but not taking part in the visit. The observer documented visit specifics including (1) physical exam performance and vitals documentation, (2) patient–provider dialogue/communication, (3) ancillary needs including laboratory, social work, nursing, imaging, or other clinical needs, (4) communication regarding the medical assessment and plan, and (5) discussion/communication with other providers present for the visit. These transcripts/observations were reviewed by a second medical reviewer, who employed deductive coding and calculated the results based on a predeveloped checklist of criteria (see Supplementary Material, Fig. 1).

Survey tools

Following their respective video visit, subjects were sent electronic versions of the Makoul Communication Assessment Tool (Makoul CAT)24 and RAND VSQ925 via REDCap. The Makoul CAT measures patient/caregivers’ perceptions of physicians’ communication skills and the VSQ9 is a visit-specific satisfaction survey. In addition, a 13-item provider survey (see Supplementary Material, Fig. 2) was developed for use in this research project via Qualtrics (Qualtrics, Provo, UT) and was distributed to providers following their first study visit. As most providers performed more than one video visit per day, surveys were not patient-encounter-specific. Free response questions included such items as “what exam components, if any, were completed in the visit,” or “what further information would have been needed to make a clinical assessment.” Providers were also queried about their comfort with telemedicine use with such questions as “In the setting of recent public health concerns, was it safer for your patient to conduct these visits by video rather than in person,” which generated binary responses (Yes/No), and percentages were tabulated. Providers gave their overall impression of the quality of the visit using a visual analog scale.

Data analysis

Analysis for the observational data included simple statistics based on the results of the deductive coding (assigning predefined codes to the qualitative data) and predeveloped criteria checklist (see Supplementary Material, Fig. 1). Analysis for the VSQ9 included overall satisfaction scores by subject evaluated in SAS University Edition (SAS Institute, Cary, NC) to examine differences in scores by subspecialty. Proportions of patients that rated their communication as “excellent” as well as overall scores for the Makoul CAT were also calculated in SAS to detect differences. Provider survey analysis included calculation of the mean and standard deviation.

Results

Study participants

A total of 40 participants, ten per specialty, were enrolled in this study. Table 1 shows mean age and sex of participants by specialty. There were a greater number of subjects who identified as female in this study, and although there were no significant imbalances of sex or age between specialties, differences in age may reflect the specialty diseases’ age preference. A range of diagnoses were seen by each subspecialty group during the course of this study and included both acute processes and chronic diagnoses (Table 2).

Table 1 Age and sex of study subjects by subspecialty.
Full size table
Table 2 Diagnoses seen by specialty.
Full size table

Observations

A total of 40 visits, ten per specialty, were observed during the course of this study. Each visit was coded, and results included the following:

  • Clinic processes: Using the observation checklist, Table 3 shows a summary of clinic processes observed during the video visits. Of the total visits, 17.5% had technical issues noted, such as patients having difficulty logging in or poor audio quality, but all visits observed in this study were completed. Technical issues may have included 12.5% of visits had clinical staff check-ins completed (e.g., by a medical assistant), and in 30% of visits the caregiver or parent provided vitals. Of the total visits, 67.5% included a focused physical exam, which included provider instruction for examination of various organ systems including skin, respiratory, musculoskeletal, neurologic, and other exam components, and completed by the patient.

    Table 3 Telemedicine visit observationsa.
    Full size table

  • Novel uses of telemedicine: The observation checklist included descriptive text regarding the use of video visits with respect to technological advancements or provider-led adaptations for telemedicine. These adaptations include clinical activities that are only able to be performed via a combination of telemedicine platform and electronic medical record (EMR) use or clinical adaptations that were completed in response to the patient’s home environment. Of note, there were no specific remote/facilitated exam tools (such as a digital stethoscope) used in these visits. The results are summarized in Table 4 and include novel tasks such as guided cast removal (using simple scissors) and “shared-screen” function use to review growth charts and labs.

    Table 4 Novel uses of telemedicinea.
    Full size table

  • Patient- vs parent-led communication: Of the 40 visits, over a third (37.5%) of the visits involved the patient leading the conversation with the provider.

Patient communication and satisfaction

The subject response rate for the Makoul CAT was 55 and 50% for the VSQ9. On the Makoul CAT the highest-rated items by the proportion of subjects who rated “excellent” as well as by the overall mean score are shown in Table 5 and included elements such as “treated me with respect,” “showed care and concern,” and “the doctor’s staff treated me with respect.” For the VSQ9, the highest-rated areas of satisfaction included “the personal manner (courtesy, respect, sensitivity, friendliness) of the person you saw” and the “technical skills (thoroughness, carefulness, competence) of the physician/health care professional you saw today.” Lowest-rated items included “getting through to the office by phone” and “how long you waited to get an appointment.”

Table 5 Highest-rated items by question on Makoul CAT and VSQ9a.
Full size table

Provider survey

The provider response rate was 91.6% (n = 22) and included providers from the four specialties. Provider reported survey outcomes assessed provider satisfaction and experience with video visits with respect to quality, clinical assessment, and patient safety during the recent public health emergency. One hundred percent of providers included in this survey felt that the video visit was a safer medium to conduct their clinical visit given the recent pandemic. In general, most providers (86.1%) identified that they were able to gather the appropriate information to make a clinical assessment and overall rated the quality of the visit highly (mean 8.1 on a scale of 1–10). In addition, 72.7% of providers reported that they were able to complete elements of the physical exam virtually. Components reported to have been completed included exams predominantly based on inspection (e.g., color, behavior, respirations, appearance of catheter site), although providers also noted specific exam features such as “parts of the neurologic exam” or “joint range of motion.” Several providers also reported measures such as blood pressure and weight. For the free response questions, the response rate was ~50% and not amenable to standard qualitative analysis. Providers had a range of answers regarding their video visit experience: some indicated positive clinical aspects of telemedicine visits (e.g., being able to observe patient in home environment), other responses included technical issues (e.g., “...virtual waiting room was a little unclear,”), and others relayed their overall feelings (both positive and negative) regarding telemedicine use.

Discussion

This study simultaneously evaluated telemedicine performance with respect to patient–provider communication, provider experience, and necessary clinical adaptations of telemedicine visits across multiple pediatric subspecialties during the COVID-19 pandemic. Amongst our findings, we determined novel uses and nontraditional applications of telemedicine during subspecialty visits that were unique to the experience of providers and patients engaging through a virtual platform as well as a construct of patients being in the home environment. We were also able to demonstrate provider acceptability during this global pandemic and have identified important areas of exploration in regard to patient communication and engagement, which are known to be imperative in patient activation, shared decision-making, and health quality outcomes.26,27,28 Prior to the pandemic, studies had indicated that on some level, telemedicine use in clinical care was acceptable to patients and caregivers,29,30,31,32 although further work on clinical outcomes and provider acceptability was needed.33 Telemedicine adoption can sometimes be difficult, particularly in pediatric subspecialty care due to concern regarding patient safety and potentially provider confidence in being able to make a clinically sound assessment by video visit. The COVID-19 global pandemic, in many ways, reduced traditional implementation barriers to telemedicine such as reimbursement issues34 allowing researchers to assess its performance and feasibility in a way that was previously challenging in this vulnerable population. Not surprisingly, this study demonstrated that given the recent public health concerns, all providers felt that it was safer for these patients to be seen via telemedicine as opposed to in-person clinical visits during the pandemic. It also demonstrated that the majority of providers believed that they were able to elicit the necessary information to conduct some level of a focused physical exam and develop an adequate clinical assessment. This was true across all four pediatric specialties. This level of confidence was encouraging given that there are few guidelines developed for best practices for ensuring high-quality video visits in pediatric subspecialty care. We found novel uses for telemedicine including guided exams and procedures, transparency of patient clinical metrics (labs, growth charts), and areas to explore with patient confidentiality (adolescent Home, Education, Activities, Drugs, Suicidality, and Sex exam). Some of these adaptations are a construct of the technology itself, while others may be due to the patient being present in the home environment, and are adaptations that could potentially be incorporated into provider education around standard practices for telemedicine use.

This study begins to explore the advances and challenges of telemedicine by pediatric specialty. It is clear that further research is important to compare video visits with in-person visits, including effectiveness in urgent, new patient, consultative visit types, effects on shared decision-making, use of adjunctive technologies for examination (e.g., auscultation), impact on health disparities (positive or negative), treatment adherence, and clinical and quality of life outcomes. Additional areas of study to consider include patient versus parent-led communication in video visits, which was observed more in the pediatric rheumatology and pediatric orthopedic surgery video visits. This is of particular importance considering that many of these visits dealt with chronic pediatric conditions, and increased patient activation and engagement are associated with improved health outcomes.35,36 Although previous studies have begun to explore the impact of screen versus face-to-face provider-to-patient communication,37 future work should explore this in pediatric care.

Limitations for this study included a relatively small sample size (both provider and patient) over a relatively short period of time to make statistically significant statements. Sampling bias also may be a limitation; the subjects participating in this study were able to successfully complete their scheduled telemedicine visit. Further large multicenter studies are needed to be able to generalize these results. This study was not designed to evaluate the reliability of specific exam components in comparison to in-person clinical visits. In addition, this study was not designed to address one of the most important barriers yet potential opportunities for telemedicine, which is improving access for vulnerable populations and decreasing health disparities due to race/ethnicity, socioeconomic status, rural communities, and disabilities. This study represents the first of its kind with understanding telemedicine use from multiple perspectives in pediatric subspecialty medicine—and although these video visits were not done due to patient-specific COVID-19-related health concerns—we have begun to be able to understand telemedicine application in high-volume, routine pediatric subspecialty care.

Conclusion

This study describes telemedicine use in pediatric subspecialties with respect to patient–provider communication, patient satisfaction, provider experience, and clinical features during the COVID-19 pandemic. It is the first study of its kind to simultaneously evaluate telemedicine performance in four different pediatric specialties and demonstrates a use case for telemedicine in follow-up pediatric subspecialty care. Although telemedicine use has skyrocketed with the pandemic, demonstrating feasibility, additional research is needed to address its acceptability and safety as part of routine pediatric subspecialty care. The hope is to be able to include telemedicine as a permanent piece in the health care delivery puzzle. More in-depth studies are needed to better understand patient and provider engagement with telemedicine, specifically, how it affects patient-reported health outcomes, patient activation, and shared decision-making in pediatric chronic care. Likewise, further research needs to be done to address novel approaches using telemedicine that may include technical interventions and provider education models. In doing so, we may begin to advance telemedicine so that we can standardize its practices, optimize provider and patient education, and improve clinical outcomes—COVID or not.