Predicting Patient-ventilator Asynchronies with Hidden Markov Models

In mechanical ventilation, it is paramount to ensure the patient’s ventilatory demand is met while minimizing asynchronies. We aimed to develop a model to predict the likelihood of asynchronies occurring. We analyzed 10,409,357 breaths from 51 critically ill patients who underwent mechanical ventilation >24 h. Patients were continuously monitored and common asynchronies were identified and regularly indexed. Based on discrete time-series data representing the total count of asynchronies, we defined four states or levels of risk of asynchronies, z1 (very-low-risk) – z4 (very-high-risk). A Poisson hidden Markov model was used to predict the probability of each level of risk occurring in the next period. Long periods with very few asynchronous events, and consequently very-low-risk, were more likely than periods with many events (state z4). States were persistent; large shifts of states were uncommon and most switches were to neighbouring states. Thus, patients entering states with a high number of asynchronies were very likely to continue in that state, which may have serious implications. This novel approach to dealing with patient-ventilator asynchrony is a first step in developing smart alarms to alert professionals to patients entering high-risk states so they can consider actions to improve patient-ventilator interaction.


Definition of patient-ventilator asynchrony and different types
Briefly, patient-ventilator asynchrony occurs when the phases of breath delivered by the ventilator do not match those of the patient. To meet the patient's demands, the ventilator's inspiratory time and gas delivery must match the patient's neural inspiratory time 1 . There are different types of asynchronies. Among the most prevalent are ineffective efforts, double cycling, short cycling, and prolonged cycling (see Supplementary Fig. S1).
Ineffective efforts are contractions of the inspiratory muscles, primarily the diaphragm, that are not followed by a ventilator breath. This asynchrony occurs when the pressure resulting from the patient's attempt to initiate a breath does not reach the ventilator's trigger threshold. In other words, the ventilator fails to detect the patient's inspiratory efforts, which are reflected physiologically by an increase in transdiaphragmatic pressure (decrease in esophageal pressure, increase in gastric pressure) and/or electrical activity of the diaphragm 2,3 . Ineffective efforts result in the patient's respiratory rate being higher than the ventilator's rate.
Double cycling consists of a sustained inspiratory effort that persists beyond the ventilator's inspiratory time, cessation of inspiratory flow, or the beginning of mechanical expiration, triggering a second ventilator breath, which may or may not be followed by a short expiration, where all or part of the volume of the first breath is added to the second breath 4 . Double cycling can cause ventilator-induced lung injury [5][6][7] . Short cycling occurs when the inspiratory time is less than one-half the mean inspiratory time, and prolonged cycling occurs when the inspiratory time is greater than twice the mean inspiratory time 4 . Inspiratory time is defined as the time during which gas flow is positive, and the mean inspiratory time is calculated over 20 cycles.

Description of the hidden Markov model
Next is a brief description of the hidden Markov model (HMM) used in our study. More general and specific details about this kind of statistical models can be found in Bishop (2007) 8 .
Sequential data can be represented as a Markov chain of latent variables, with each observation conditioned on the state of the corresponding latent variable. An HMM assumes that observations are generated by different probability distributions corresponding to the discrete multinomial latent variable. In the setting of this study, the hidden states can be interpreted as proxies for patients' level of synchrony with the ventilator; each state can be associated with a different frequency of events and therefore results in a different level of risk.
Thus, HMM can detect states with different frequencies of events, so it can predict the number of events that will occur in a period. HMM automatically detects whether a patient is at a 'low-risk state' (low frequency of events) or at a 'high-risk state' (high frequency of events). The number of states is a parameter that needs to be set by the user before training the model. Given any number of possible predefined states, the model finds the most probable distribution for each state, a posteriori, and also makes it possible to detect when the patient changes from one state to another. Then, the uncertainty of being in each state, represented by this posterior probability distribution, can be summarized in terms of credible intervals.

The hidden Markov model
In the context of an HMM, is defined as the number of events during the period and is the state associated with that period. There are states, each of which has different characteristics. At period , all states have a probability of occurring, but only one of them actually occurs.
An HMM has two main components: transition probabilities and emission probabilities.

Transition probabilities
The relationship between period and period + 1 in the HMM is governed by the transition matrix, , which represents the probability of switching from one state to another. If there are states, has dimension × and its elements , represent the probability of switching from state to state . Since the elements of the matrix are probabilities, their values must be between zero and one, and the sum of each row in the matrix must equal 1.
If the diagonal elements in are much larger than the off-diagonal elements, then the data sequence will have long runs of points generated from a single state and infrequent transitions from one state to another; if the diagonal and off-diagonal elements are similar, the state of the sequence will change frequently. Thus, the values of determine how persistent the states are.

Emission probabilities
The distribution of the observation given the state is called the emission probability, denoted as ( | , ). The different hidden states are associated with different probability distributions with different parameters . At period t, the observation is generated by one of the k possible probability distributions, depending on the state of that period and the set of parameters that determine the chosen probability ditribution. In this study, the chosen probability distribution was the Poisson distribution, which is the most common choice for event counts 9 . Thus, is the parameter of the Poisson distribution, which represents the expected count of events at the state k.
Therefore, if the model consists of different states, the emission probabilities of those states are Poisson distributions with different parameters λ.
In HMM the data generating process works as follows: At period , the hidden state has a certain probability of taking on any of the possible predefined states, and this probability depends on the state in period − 1. These probabilities come from the corresponding values in the transition matrix . Once a state is achieved, the observation is sampled from that state's probability distribution, which is the emission probability. Then, at period + 1, the probabilities of the possible states depend on the state that was achieved in period . Once the state +1 is achieved, the observation +1 is sampled from the distribution associated with that state. This process continues indefinitely until the last period. The first state of the process, given that it is does not have any previous state, is usually sampled from a distribution where all states have equal probability.

Estimation
The expectation maximization algorithm iteratively computes the transition and emission probabilities 10 . We initialize this algorithm with random values for the transition and emission probabilities. Next, the Viterbi algorithm 11 uses the emission and transition probabilities estimated earlier to find the most likely sequence of the latent states (i.e., the posterior probability distribution) that generated the data.
In many applications of hidden Markov models, the latent variables have some meaningful interpretation, and so it is often of interest to find the most probable sequence of hidden states for a given observation sequence. However, finding the most probable sequence of latent states is not the same as finding the set of states that are individually the most probable. The set of states that are individually the most probable will not correspond to the most probable sequence of states, because the sequence also depends on the transition probabilities. In fact, if the two successive states that are the most probable individually are connected by a transition matrix element whose value is zero, the probability of the sequence will be zero.

Prediction
At any period , once the state is achieved, it is possible to predict the expected value of +1 from the estimates of the transition and emission probabilities. The prediction of  Table 1 (main text) and Supplementary Table S2.

Validation
To validate the model, we used a k-fold cross validation procedure. Five different training/validation subsets were randomly selected, so that all patients were used for both training and validation, and each patient was used just once for validation. Due to the amount of data and the complexity of the algorithm, the number of folds is limited by computational power. Following this limitation, the selected number of folds was set to five. The model's predictive ability was assessed in terms of a root mean squared error.