Serious illness communication (SIC) is an essential component of palliative care that ensures the delivery of goal-concordant care. SIC is often defined as the conversations between clinicians and patients with serious illness about their goals, values, and priorities1. High-quality and timely SIC enables and enhances decision-making and care planning through the process of cultivating patients’ prognostic awareness and translating their values and priorities into patient-centered recommendations. The iterative and non-linear process of SIC requires frequent and early conversations to ensure that clinicians accurately understand patients’ evolving goals, values, and priorities and to make patient-centered recommendations throughout the illness trajectory.

The traditional SIC delivery process consists of a series of conversations where gathering, interpreting, and integrating SIC data occur within a clinical encounter followed by manual clinician documentation in the electronic health record (EHR) post-visit. This process can be broken down into the following steps: determining patient eligibility for SIC; gathering and interpreting information (e.g., eliciting and clarifying the patient’s illness understanding, hopes, and worries); conducting a therapeutic conversation (e.g., counseling and supporting the patient on coping with life-threatening illness) with the goal of shared decision-making; documenting the conversation; and making SIC documentation accessible to others in the EHR (Fig. 1). However, each step is a potential bottleneck because the ability to initiate SIC or make forward progress depends heavily on the clinician’s ability, skill, and judgement. This is problematic for several reasons.

Fig. 1: Human versus hybrid artificial intelligence (AI)-human workflow for serious illness communication (SIC).
figure 1

The current workflow relies on human judgment to identify SIC-eligible patients and manual effort to initiate SIC, to document SIC, and to locate SIC documentation in the electronic health record (EHR). A hybrid AI-human workflow would leverage AI to identify SIC-eligible patients more accurately and to streamline the workflow by helping complete essential menial tasks, thus ensuring more seriously ill patients will receive timely SIC and allowing clinicians more time and energy to focus on the higher-order cognitive and emotional tasks, including problem-solving. Natacha Meyer designed and illustrated the figure and provided permission to use this figure in the manuscript.

Full size image

First, most clinicians lack SIC training and feel unprepared to have these difficult conversations with their patients2. Second, patients and/or clinicians may be unclear about the optimal timing and when to make such conversations a priority2. Third, clinicians often lack time to conduct SIC2 and to document these conversations adequately3. Fourth, standards for EHRs to facilitate consistent, accurate documentation that is easily accessible to all care team members are lacking4. Therefore, in addition to training more clinicians to be competent in SIC, a novel workflow that addresses these barriers will be necessary to ensure that all seriously ill patients receive timely and effective SIC that informs their care in real time and naturally results in documentation of patients’ goals and preferences that is visible to others. We propose that a hybrid artificial intelligence (AI)-human workflow can improve this process by helping clinicians identify patients with SIC needs more accurately; promoting upstream data collection to facilitate more efficient in-person shared decision-making; reducing clinician documentation burden by streamlining the SIC documentation process; facilitating seamless sharing of patient goals and preferences via accurate and efficient identification of SIC documentation in the EHR; and providing real-time feedback to clinicians on their SIC skills.

Patients with serious illness often experience delayed SIC because clinicians are poor at prognosticating life expectancy for terminally ill patients, usually erring on the side of optimism5. Moreover, systematic methods to identify patients with palliative care needs are lacking6. To solve this problem, AI researchers have developed machine learning algorithms to generate more accurate mortality predictions to facilitate earlier SIC and palliative care delivery7,8. Some researchers have demonstrated that coupling AI-generated mortality predictions with behavioral nudges to clinicians can improve SIC frequency9. However, critics have expressed worry about using mortality predictions alone for identifying populations with palliative care needs because a reductionistic interpretation of these results may lead to further propagation of algorithmic or other systemic biases leading to inequitable care and patient harm10. Therefore, others have suggested alternative metrics that identify patients at risk of worsening serious illness to train predictive algorithms—including functional decline, deteriorating quality of life, escalating caregiver burden, or psychosocial or spiritual distress10. Some accountable care organizations are already using claims-based algorithms to identify high-cost patients who would benefit from earlier palliative care11, but greater effort is needed to mitigate algorithmic bias, especially among commercially available products that are widely used12. Moreover, additional methods to identify SIC-eligible patients should be considered since EHR-based algorithms often have performance gaps13.

The use of conversational agents (aka chatbots) has largely been unexplored in palliative care. Conversational agents that are emotionally aware or use unconstrained natural language input are nascent in health care14,15,16, but the technology to date is mature enough to support its use in SIC as a basic data-gathering agent. One could imagine SIC conversational agents that would collect information about the patient’s prognostic awareness and priorities prior to in-person visits. Doing so would enable clinicians to maximize face-to-face time on higher-order cognitive and emotional tasks (e.g., interpreting patient preferences and responding empathically to a patient’s emotional state) that would lead to earlier shared decision-making. Conversational agents may also give patients time to reflect and discuss issues with trusted persons prior to meeting with the clinician.

To date, no studies on conversational agents have been conducted on patients with serious illness, but proof-of-concept studies in the general population have demonstrated the acceptability of conversational agents that address palliative care-related topics17,18. In one study, older adults utilized multiple-choice responses to converse with an agent that provided spiritual counseling, which reduced anxiety and increased the intent to create a last will and testament17. In another, machine learning algorithms allowed the agent to collect patient-reported outcome measures and display empathy to the users’ free text responses18. Although conversational agents were well-received in these preliminary studies, some patients will prefer to have the entire SIC with their clinicians directly, obviating the need for a chatbot. Further studies need to be conducted on actual patients with serious illness and should assess if conversational agent-led SIC triggers emotional distress in patients or actually enhances the patient-clinician relationship. Moreover, deploying conversational agents may inadvertently widen inequities in certain populations, particularly patients with limited English proficiency, health information technology literacy, or broadband access.

AI can also streamline the SIC documentation process and potentially improve the quality of SIC documentation via natural language processing (NLP)—a form of machine learning designed to understand, interpret, or manipulate human language. Missing or incomplete documentation in the EHR regarding patient preferences for life-sustaining treatment is common and contributes to medical errors related to end-of-life care3. NLP-enabled dictation software has demonstrated the ability to reduce medical documentation time while maintaining documentation quality19 and is already commercially available20. Such technology would reduce the time clinicians spend manually writing notes and minimize recall bias since the content of the conversation is transcribed verbatim during the conversation and not hours later, typical of much documentation. As a result, nuanced details of the conversation are readily captured in real time leading to higher quality notes with less clinician effort.

NLP also has the potential to address barriers resulting from poor EHR design that prevent or inhibit the extraction and flow of meaningful advanced care planning information across the care continuum4. In its current state, identifying SIC documentation in the EHR typically involves a manual chart review that possibly includes a keyword search or utilization of note filters. NLP-enabled software that identifies free text SIC documentation would likely reduce the time and effort clinicians spend looking for this information and prevent inadvertent oversight of patient preferences leading to goal-discordant care. AI-assisted chart reviews have demonstrated higher accuracy and shorter time for extracting relevant patient information compared with standard chart reviews21. Additionally, NLP has demonstrated the ability to identify SIC documentation accurately from EHR data and, in some cases, more accurately than human coders22,23,24. However, the accuracy of NLP to identify SIC documentation largely depends on the quality of the gold standard dataset created by human annotation used to train the model. Consequently, widespread implementation of NLP-enabled software to identify SIC documentation likely remain years away since high-quality annotated examples to train generalizable models are lacking, and adapting NLP models between different datasets often require additional training or fine-tuning25. In the interim, some health systems have created a centralized location for SIC documentation in the EHR to improve SIC documentation identification9,26, but compliance with utilizing these modules will likely remain an issue and additional NLP assistance will optimize the identification of SIC documentation in the EHR27.

Finally, AI has the potential to improve SIC delivery by providing speech analysis and personalized feedback to clinicians regarding their communication skills28. Automated speech analysis and communication feedback will likely take years to manifest because not only do technical and logistical barriers remain (e.g., lack of adequate high-quality SIC recordings to accurately assess non-linguistic features)28, but also greater consensus is needed to define and measure basic communication quality and outcomes29. Researchers are currently utilizing NLP to analyze audio recordings of SIC to characterize and understand the naturally occurring features of these complex conversations30,31,32, such as identifying intentional pauses that foster empathy, compassion, and understanding, aka “Connectional Silences.30" This type of research will guide future efforts to develop ways of automating the measurement of SIC quality in real time, allowing for immediate feedback to improve clinician performance.

In conclusion, a hybrid AI-human SIC workflow may improve the efficiency and effectiveness of SIC delivery in busy practice settings. Some of the AI technology are available for widespread use presently (e.g., risk prediction algorithms and NLP-enabled transcription software), whereas others are emerging technologies that are being developed and studied (e.g., SIC conversational agents and NLP-enabled identification of SIC documentation). This proposed paradigm still requires that clinicians undergo some SIC training to capitalize on the assistance provided by AI, as well as additional research to avoid unintended consequences of AI implementation. That said, a semi-automated approach to SIC delivery holds tremendous promise and would likely improve current SIC workflow by optimizing clinical manpower and efficiency while increasing the likelihood that these critically important conversations will occur effectively and in a timely fashion.