simKAP: simulation framework for the kidney allocation process with decision making model

Organ shortage is a major barrier in transplantation and rules guarding organ allocation decisions should be robust, transparent, ethical and fair. Whilst numerous allocation strategies have been proposed, it is often unrealistic to evaluate all of them in real-life settings. Hence, the capability of conducting simulations prior to deployment is important. Here, we developed a kidney allocation simulation framework (simKAP) that aims to evaluate the allocation process and the complex clinical decision-making process of organ acceptance in kidney transplantation. Our findings have shown that incorporation of both the clinical decision-making and a dynamic wait-listing process resulted in the best agreement between the actual and simulated data in almost all scenarios. Additionally, several hypothetical risk-based allocation strategies were generated, and we found that these strategies improved recipients’ long-term post-transplant patient survival and reduced wait time for transplantation. The importance of simKAP lies in its ability for policymakers in any transplant community to evaluate any proposed allocation algorithm using in-silico simulation.

There are a limited number of simulation frameworks for allocation algorithms in the literature, with the most well-established model being the Simulated Allocation Model (SAM) 11 , which was initially developed for liver transplantation but was later expanded to include kidney and pancreatic transplantation (SRTR).In addition to SAM, several other countries including the Netherlands based on the Eurotransplant system 12 , and Spain 13 , have also developed similar algorithms specifically to capture the allocation process in their country.Most of these simulation frameworks are not adaptable to the input of another proposed allocation rule and it is focused on a specific allocation system only.Other work in simulation consists of post allocation acceptance process 14,15 , models with a special focus on candidate list generation 16,17 and algorithms 18 that primarily simulate the actual allocation rules.However, these methodologies do not include mechanisms for capturing the increasingly prevalent use of shared decision-making in donor-kidney acceptance.
In this study, using deceased donor kidney transplantation as an example, we developed a flexible simulation framework for the Kidney Allocation Process (simKAP) that considers both the dynamic changes of the candidate waiting lists and a joint decision-making process between the candidates and clinicians.This framework is not organ specific and therefore, it is also generalizable to other non-kidney related allocation challenges.In this study, we evaluated the performance of our simulation model using data from the Australian and New Zealand Dialysis and Transplant (ANZDATA) Registry, demonstrating that simKAP better reflects the reality and as such, offers the opportunity to deepen the understanding of different kidney allocation processes.We further illustrated the potential of simKAP to guide policies that aimed to mitigate inherent allocation bias and adhere to agreed societal values.

Methods Dataset
Data from the ANZDATA registry was used for the modelling of simKAP.This data consists of all Australian candidates (n = 7740) on the waiting list who received a matched deceased donor kidney with starting replacement therapy time (KRT) between 30th June 2006 and 13th November 2017.Multiple kidney transplantations were included, but multi-organ transplantations were excluded.The deceased donor waiting list data was obtained from the ANZDATA 40th Annual report by the National Organ Matching System (NOMS) 19 .

Simulation of Kidney Allocation Process (simKAP)
The simKAP framework consists of three stages: (A) generation of the candidate list, (B) creation of the allocation rule, and (C) development of a shared-decision-making.The simKAP framework is flexible and allows for the inclusion of other allocation algorithms as an alternative; we illustrate this by providing two hypothetical allocation algorithms.The simulation requires two specific data inputs (Fig. 1): first, candidate clinical information, and second, donor clinical information.The simulated donors were first generated by selecting a random sample of simulated donors (default size of random sample is 800) and sorted chronologically according to the transplant date.The simulation procedure simKAP then sequentially took each simulated donor individually to generate a corresponding simulated recipient.

Phase A: Candidate list generation
Based on a given candidate clinical data, the simulation procedure, simKAP first generated 300 initial candidates denoted by the set W 0 = {w 1 ,…,w 300 } and the notation W t represents the set of individuals that were on the waiting list at time point t.The initial number of candidates on the waiting list were chosen such that the ratio in simKAP was similar to the ratio of the annual number of disease donors (DD graft) and the annual number of candidates "made active" and "taken off the list" from the Australian Transplant waiting list between 2011 and 2016 (Table 6.1 of the ANZDATA 40th Annual Report 2017) 19 .Next, a stratified sampling strategy was implemented for the selection of appropriate candidates for waiting listing.Under this stratified sampling strategy, weights were selected such that the simulated waiting list distribution reflected the actual waiting list distributions.The four individual characteristics taken into consideration were recipient age, blood groups, residing states of the candidates and sensitization levels.These variables were selected based on clinical knowledge that these are known risk factors for post-transplant allograft and patient outcomes.
The arrival to the candidate waiting list and exit from the candidate waiting list without transplant were modelled using Poisson processes.Here, a Poisson process queuing model with the Poisson rate parameter λ in (default λ in = 0.7) representing the daily arrival rate of waiting listed candidates was used to simulate the number of incoming recipients and their corresponding simulated waiting list start date.Next, a Poisson process queuing model, with the Poisson rate parameter λ out (default λ out = 0.4) representing the daily departure rate, was used to model the number of potential candidates being put on the interim list for a number of reasons including acute illnesses and hospitalization.To select the designated recipients leaving the waiting list without a transplant, we first estimated likelihood of temporary departure from the waiting list by calculating the probability of "exit without transplant" for all individuals currently on a waiting list using a Random Survival Forest (RSF) model with eleven recipient features, including recipient age, gender, lung disease, smoking status, diabetes, cardiovascular disease, and cancer condition.Then, n out , the number of recipients exited from the list without a transplant, was obtained based on a Poisson model with mean λ out .The n out individuals with the highest exit risk score were then removed from the current waiting list W t .In addition, at a given time t, a dynamic rate parameter k in (t)λ in and k out (t)λ out instead of λ in and λ out , respectively, was used in the model.This allowed for varying rates across different time periods and captured the yearly fluxes or trends in daily arrival or departure rates.
• National-state-national allocation rounds: where the first national-state round follows the ABO-strict match for blood group, and if the donor is not matched, the second national round will attempt to match a donorrecipient pair under a more general ABO-compatible match (Table 1).In the current simKAP package, a corresponding list of allocation scores of the candidate list was matched to every incoming donor kidney through a shared decision-making process and created the final donor-recipient match (or whether to accept or reject a proposed match).The four "consideration variables" that were included in this framework for the jth individual are: • Pediatric status: α 1j = I(age j < 18 and |age j -age donnor |< 30), where I(•) denotes the indicator function which has value one if the condition is satisfied and is zero otherwise; • Donor and recipient quality measured by kidney donor risk index (KDRI) and recipient estimated post- transplant survival (EPTS): α 2j = EPTS j −KDRI donor ; • For recipients having blood type AB, they are more likely to accept a kidney from the same blood type.: α 3j = I(ABO j = AB and ABO donor = AB); and • Donor and recipient state to account for practical benefits of not needing to transport kidneys across interstate borders: Using the simKAP framework, we implemented two shared decision-making models (sdm0 and sdm1) to capture clinician-patient decision-making in the donor-recipient matching process.
The first (sdm0) is a matching process which selects the donor-recipient pair with the highest allocation score.
The second (sdm1) is based on a nested decision tree process (Fig. 1b).We first re-order all the eligible candidates by descending order based on allocation scores calculated in Part B of simKAP.We then attempted to match a given j individual with the donor under α ij , i = 1, 2, 3, or 4. When attempts under all four considerations had been exhausted, and a donor-recipient match was not found, simKAP then sought to match the (j + 1)st individual with the donor.
The acceptance probabilities for all four considerations were modelled by a Bernoulli distribution with acceptance probability p j , which differs according to different recipient's characteristics (PRA and HLA).For a typical recipient j, the weighted human leukocyte antigen (HLA) mismatch is defined as HLA j = 2HLA DRj + HLA_ A j + HLA_B j , where HLA_DR j , HLA_A j and HLA_B j represent recipient's j mismatch counts from HLA-DR, HLA-A and HLA-B, respectively.This weighted sum aims to capture the different importance the allocation community currently places on HLA-DR versus HLA-A and HLA-B, respectively.This aspect can be customized.The PRA score is a percentage value between 0 and 100 capturing the sensitivity of an individual to HLA antigens.We use a transformed value (1−PRA j /100), such that a value close to 0 represents an individual with high sensitivity.The acceptance probability p j takes the product of two transformed scores and is a value between 0 and 1.This is written as p j = (1−PRA j /100)(1−HLA j /z), where z is a tuning parameter allowing users to place different emphasis between sensitivity and HLA mismatch.Our current default choice for z is 50.An individual with high sensitivity (small (1-PRA j /100) value) and high mismatch (small (1-HLA j /z)) is less likely to accept.

Alternative allocation algorithms
We then implemented the two alternatives of risk-based allocation algorithms (B1 and B2) as detailed in Table 2.Both algorithms applied risk-based matching system that involved preferential matching of the recipient and donor pair based on the Kidney Donor Profile Index (KDPI), a measure of the donor kidney quality (the lower KDPI the better the quality) with the EPTS measuring the recipient health condition (the lower EPTS the better  21 .These scores are moderately good at discriminating post-transplant survival of adult kidney transplant recipients as well as allograft survivals. [B1] Cut-off Risk based allocation process (CORisk allocation).For a given risk-cut-off number c , the can- didates with EPTS scores of c% or less were offered kidneys from donors with KDPI scores of % or less and the candidates with EPTS scores of % or more received offers from donors with KDPI scores of c% or more.Our candidate EPTS values were recalculated as where I("Recipient age > 25") and I("Years on dialysis = 0") are both indicator functions.The CORisk was implemented in the function 'selection_corisk' .
[B2] Interval Risk based allocation process (IRisk allocation).For a given risk-cut-off number s , the candi- dates with EPTS scores within a given bandwidth received offers based on the KDPI scores..The EPTS bandwidth for a given KDPI score was defined as the interval between (KDPI-s%) and (KDPI + s%).IRisk was implemented in the function 'selection_irisk' .
The implementation of simKAP is made available in the statistics software R through a package available at https:// github.com/ Sydne yBioX/ simKAP.The package allows for the simulation of multiple scenarios through modification of each of the function arguments of simKAP.User details can be found in the package vignette.

Performance validation
Three processes were compared in this performance evaluation: Process I: Maximum score selection without dynamic waiting list and shared decision-making (Phase B); Process II: Shared decision-making without dynamic waiting list (Phases B and C); Process III: Both shared decision-making and dynamic waiting list (Phases A, B and C).
We have developed an "Allocation Characteristics Comparison Analysis" (ACCA) workflow to comprehensively validate the three processes of simKAP based on allocation data characteristics between the actual and simulation results for the following allocation characteristics: 1. Ratio of National-to-State based allocations.2. Recipient waiting time.

Percentages of donor kidneys allocated under different States.
For each of the above-mentioned quantities, we used the metrics listed below: • The difference between the two values.
• The median difference between the two values after stratification.
• The Hellinger distance between the two values, which calculates the distance between two categorical vectors.
The smaller the Hellinger distance the closer the characteristics.• Kolmogorov-Smirnov (KS) statistic and the kurtosis statistic between the two values, which quantify the similarity between two distributions.The smaller these values, the more similar the two distributions.

Ethical approval and informed consent
There is no ethics requirement for this as this is a de-identified registry dataset.

Results simKAP presents a simulation framework for the Kidney allocation process
Figure 1a shows the simKAP, a flexible and dynamic simulation framework, that incorporates a shared decisionmaking process for deceased donor kidney allocation.The three stages are (A) generating potential transplant candidates, (B) rule and definition for computing a utility-based allocation score, and (C) selecting donor-recipient couples through decision-making.In part A, simKAP generated a list of transplant candidates at a specific time using the Poisson processes to dynamically model the arrival and departure of model candidates.In part B, the allocation algorithms (current or proposed) automatically granted an allocation score for all transplant candidates with a given simulated donor.In part C, a shared decision-making process was evaluated and observed resulting in a selected recipient for a particular donor.A shared decision-making process is important to both healthcare professionals and patients when discussing the concepts of kidney allocations and transplantation, since there are many different options, competing concerns and risks that are unique to transplant recipients 22 .Such a decision requires a partnership between health professionals and patients with action required from all parties.For example, the factors that patients and health professionals prioritize in accepting or declining a deceased donor kidney offer are likely to be different.Prior study has reported patients placed the greatest value on kidney quality and predictors of transplant outcome 23 .Therefore, we have included donor and recipient quality measured by KDRI and EPTS along with three other variables to capture clinician-patient decision-making in the donor-recipient matching process (see Methods).
Raw EPTS =0.049 × Recipient age × I Recipient age > 25 + 0.493 × Prior kidney transplant + 0.287 × log Years on dialysis + 1 + 0.598 × "Years on dialysis = 0 ′′ , Together, simKAP was built with the capacity of replicating and reflecting real-life allocation processes.Users can customize the individual components within the simKAP package.

simKAP's architecture reflects observed national-to-state allocation proportions
To establish the most realistic allocation procedure, we compared three different allocation processes that captured the various combinations of the three parts of the allocation processes using the independent validation dataset, as detailed in Fig. 2a.We anticipated that the realistic allocation procedure in a two-tier allocation system would reflect the observed proportion allocated under the national scheme.In Australia, approximately 25% and 75% of deceased donor kidney transplants were allocated using the national and state allocation schemes, respectively.The process I represents a simulation algorithm that focuses solely on the rules and definition assigned by the allocation system (Phase B), Fig. 2a (top-right panel) shows a (45:55) national-state allocation ratio which does not reflect the actual value (25:75; Fig. 2a top-left panel).We then examined another two processes that incorporated shared decision-making (Process II) and all three phases (Process III) in the allocation process which show improved performances.

Comprehensive assessment demonstrates simKAP's capacity to capture actual results
We then validated our predictions against the actual results and compared several allocation characteristics through the calculation of a dissimilarity score between simulated and actual results characteristics (see Method, Performance validation).Overall, Process III had the best performance as illustrated by having the smallest dissimilarity score (Fig. 2b) across recipient waiting time, donor state and national-state proportions.Moreover, the summarized dissimilarity score decreased from 0.3 to 0.15 when comparing results from Process III and Process I. Figure 2c   To illustrate simKAP's potential to inform policy makers in allocation algorithms, we simulated additional hypothetical allocation algorithms and observed the impact on how the deceased donor kidney was allocated among the different transplant candidates.We implemented two hypothetical types of risk-based allocation algorithms (Fig. 3a): CORisk and IRisk processes (see Method: Alternative allocation model).Both risk-based allocation procedures were inspired by the current algorithm for US new KAS Kidney transplant allocation system 24 .
Next, we examined the simulated results of five different allocation processes detailed in Table 2. Figure 3b shows the effects of the risk-based allocation procedure, where the KDRI and EPTS values for each simulated donor-pair are displayed in each scatter plot.The existing method, as predicted, does not explicitly account for donor or beneficiary quality, hence no specific pattern was observed.In the remaining four scatter plots, the restriction of the various KDRI and EPTS thresholds highlighted the allocation restriction that was implemented as part of the risks-based algorithm.For example, with CORisk(0.4)model, the candidates with EPTS scores of 40% or less received offers for kidneys from donors with KDPI scores of 40% or less and the candidates with EPTS scores of 40% or more will receive offers for kidneys from donors with KDPI scores of 40% or more.The scatter plot shows that no allocation will happen between a donor with KDRI 0.5 and a candidate with EPTS 0.2.
As expected, the waiting period varied by age group, and the estimated post-transplant survival probability decreased with increasing age.In the current illustration settings, we have demonstrated (Fig. 3c) that the five different algorithms had relatively minimal influence (similar distributions) on the recipient waiting time and 10-year survival probability for candidates of various ages.Furthermore, for highly sensitive patients, the selected risk-based allocation algorithm (CORisk_0.4)provided recipients with higher quality kidneys and reduced waiting time (Supplementary Fig. 1).An overview of the comparisons between the five allocation algorithms are provided in the Supplementary Material.This demonstrates the flexibility of simKAP as a tool to examine the impact of multiple proposed allocation algorithms using in-silico simulation.Additional case studies are provided in the Supplementary Material.

Discussion
In this novel simKAP simulation model, we used a series of allocation rules to simulate the entire process of the deceased donor kidney allocation in Australia.This model incorporated a decision-making process that takes into consideration the uncertainties of clinical decision-making in real-life settings and evaluated the downstream effects of the simulation process.Through the inclusion of the novel element of shared decision modelling, we found the simulated results from simKAP better reflect real life data.
Binary classification modelling approaches, such as logistic regression or support vector machine, have been used to predict whether a candidate will accept a given kidney or not in the literature 14 .However, the logistic regression model is dependent on specific regional information and should be built specific to each country.Here, Phase C of shared decision-making in simKAP proposes several models to capture the behavioural variabilities between clinicians and health professionals, and other models could be applied to capture behavioural variabilities in other jurisdictions, including any user-defined logistic regression models.The choice of the starting number in the waiting list (W 0 ), together with λ in and λ out , will have an impact on the waiting list distribution throughout the simulation process.A higher rate of arrival and lower rate of departure will result in more transplant candidates on the waiting list.In contrast, a lower rate of arrival and higher rate of departure will result in less transplant candidates on the waiting list.Our current default setting was created such that the ratio of number of kidney grafts per year to the number of candidates arriving and departure on the waiting list was similar to the number of transplant candidates being delisted from the deceased donor waiting list.When designing the simulations for modelling the expected size of the waiting list, it is important to jointly determine the three parameters.
Computing speed is critical when it comes to establishing simulation processes, and the computation cost is typically generated by the desire for being able have many repetitions.The ability to repeat the simulation a suitable number of times to capture the variability of simulated recipient-donor pairs, contributes to the stability and reliability of the simulation findings.Hence, our simKAP model addresses the computational challenge by integrating parallel processing across many cores, thus considerably increases simulation speed.For example, resampling 20 times on a standard personal computer took 1745s (around 30 min) without parallelization whereas this reduced to 847 s (around 14 min) when we parallelized the same simulation using two cores.Our approach can distribute replications across all cores of that the device can handle concurrently rather than sequentially and thus decreases the overall computational load.
The simKAP model has many relevant clinical and policy implications.Transplant professionals are entrusted with the stewardship of deceased donor organs.Transplant clinicians are accountable for making the most appropriate decisions to ensure the donor organs are used equitably and efficiently to maximize survival gains and quality of life for those deemed suitable for transplantation.Therefore, the organ specific allocation policy should include the appropriate performance indicators, and a self-evaluation process to ensure the data and outcomes are aligned with the performance goals.The simKAP framework allows the assessment of the highly dynamic nature of the deceased donor kidney waiting list, and has the capability to evaluate these changes realtime, assuring the public that the ethical principles organ donation and allocation are upheld, and the desired outcomes are achieved.
The current limitation of simKAP is in its capacity to handle multiple transplantation such as dual kidney-liver or kidney-pancreas transplantation.To establish such an algorithm, the simulation process can be extended to incorporate complex input that handles both the offer and acceptance of multiple organs.In addition, simKAP can be further validate in other jurisdictions.Going forward, simKAP could be repeated applied to estimate individual dynamic profile of potential transplant offers and extended to handle complex allocation scenarios which will shape future research priorities.

Conclusion
Here we developed a simulation framework for the entire Kidney Allocation Process (simKAP) of deceased donor that incorporated a novel element of shared decision-making modelling that capture uncertainties of clinical decision-making in real-life settings.Our detailed evaluation illustrates that in many scenarios, the incorporation of both the clinical decision-making and a dynamic wait-listing process provides the best agreement between the actual and simulated data.The application of the simKAP model is broad and has the potential to influence the allocation and acceptance decisions made by the transplant health professionals and the patients.

Figure 1 .
Figure 1.SimKAP Schematic of our simulation model.(a) simKAP workflow.simKAP takes kidney transplant candidate and donor data as inputs and outputs the new simulated recipient-donor pairs.Three phases implemented in simKAP are illustrated from the top panel to the bottom panel: Candidate list generation; Allocation rule; Shared decision making.(b) Decision making algorithm based on the sdm1 model.

Figure 2 .
Figure 2. Comparison of different allocation simulation processes.(a) Comparison of kidney allocated proportions via the national and the state algorithm.(top left: Actual proportion, top right: Process I, bottom left: Process II, bottom right: Process III) (b) Dissimilarity scores for allocation characteristics, smaller scores indicates better performance of simulation (left to right: recipient waiting time, donor state and national-state proportion) (c) Heatmap of median recipient waiting time |log(actual/simulated)| for different recipient groups (left to right: States, Age, EPTS, PRA, Blood group); a small value close to 0 indicates good similarity between actual and simulated values.Black stars indicate the best process in each category.

Table 1 .
Blood group compatibility match rule illustrates possible non-strict match.

Table 2 .
Hypotheticalthe health condition).The EPTS and the KDRI scores have both been externally validated in the Australian kidney transplant populations allocation algorithms and their corresponding compositions in terms of eligibility (A), allocation rule (B) and decision making (C).Vol.:(0123456789) Scientific Reports | (2023) 13:16367 | https://doi.org/10.1038/s41598-023-41162-wwww.nature.com/scientificreports/ further illustrates the superior performance of Process III with the stratification of the median waiting time by categories of recipient characteristics and included recipient EPTS, recipient PRA level, recipient age groups, recipient blood group and the donor state.We observed Process III had the smallest values (the best) for all (20 out of 21) except for pediatric recipients (age < 18) group.