Development and validation of a simplified risk prediction model for preterm birth: a prospective cohort study in rural Ethiopia

Preterm birth is one of the most common obstetric complications in low- and middle-income countries, where access to advanced diagnostic tests and imaging is limited. Therefore, we developed and validated a simplified risk prediction tool to predict preterm birth based on easily applicable and routinely collected characteristics of pregnant women in the primary care setting. We used a logistic regression model to develop a model based on the data collected from 481 pregnant women. Model accuracy was evaluated through discrimination (measured by the area under the Receiver Operating Characteristic curve; AUC) and calibration (via calibration graphs and the Hosmer–Lemeshow goodness of fit test). Internal validation was performed using a bootstrapping technique. A simplified risk score was developed, and the cut-off point was determined using the “Youden index” to classify pregnant women into high or low risk for preterm birth. The incidence of preterm birth was 19.5% (95% CI:16.2, 23.3) of pregnancies. The final prediction model incorporated mid-upper arm circumference, gravidity, history of abortion, antenatal care, comorbidity, intimate partner violence, and anemia as predictors of preeclampsia. The AUC of the model was 0.687 (95% CI: 0.62, 0.75). The calibration plot demonstrated a good calibration with a p-value of 0.713 for the Hosmer–Lemeshow goodness of fit test. The model can identify pregnant women at high risk of preterm birth. It is applicable in daily clinical practice and could contribute to the improvement of the health of women and newborns in primary care settings with limited resources. Healthcare providers in rural areas could use this prediction model to improve clinical decision-making and reduce obstetrics complications.


Study setting and design
We used data from the Butajira Nutrition, Mental Health, and Pregnancy (BUNMAP) project among pregnant women and their newborns in Southern Ethiopia.The BUNMAP project was established in 2016 under the Butajira Health and Demographic Surveillance Site (BHDSS), consisting of nine rural and one urban dweller association.The BHDSS is one of the oldest Demographic and Health Survey sites in the Southern Ethiopia region 41 .
The BUNMAP project was an open, prospective cohort of pregnant women and their offspring (up to 59 months) collected between 2017 and 2019.We developed a prediction model for preterm birth using the baseline characteristics of pregnant women and birth history.The theoretical design was the incidence of preterm birth as a function of multiple predictors of ambulatory pregnant women.Health extension workers in the BHDSS identified pregnant women aged 15 to 49 through house-to-house surveillance.

Study population
At baseline, all pregnant women aged 15-49 years with gestational age between 8 and 24 completed weeks, living in the Butajira HDSS, and planning to deliver in the study area were included in the study.Pregnant women who were severely malnourished or had a mid-upper arm circumference (MUAC) of less than 17.5 cm and severely anemic women with hemoglobin (Hgb) levels less than seven g/dL were excluded from the cohort.Our study included pregnant women enrolled in the cohort who met the inclusion criteria and had a well-determined gestational age at birth (Fig. 1).

Predictors and outcome assessment
After enrolment, all women were requested to travel to the nearest health facility for a comprehensive baseline assessment.
An experienced sonographer used transabdominal, portable diagnostic imaging, and full-color flow mapping ultrasound to confirm gestational age at baseline.The primary outcome, preterm birth, was defined as birth before 37 completed weeks of gestation.The study questionnaire was adopted and developed from the validated tools and Ethiopian demography and health survey to collect data on the following predictors: Intimate partner violence (IPV), maternal stress, maternal age, marital status, educational status, pregnancy type, substance use, Mid-upper arm circumference (MUAC), Comorbidity, history of previous and current Antenatal care (ANC) visits, history of abortion, history of preterm birth, gravidity, contraceptive use, deworming, and comorbidity.Then, potential predictors were categorized by considering the clinical relevance thresholds for adverse birth outcomes.
Mid-upper arm circumference was measured using a standard MUAC tape, and less than 23 cm was considered maternal malnutrition 42 .
Intimate partner violence was assessed using the Hurt, Insult, Threaten, and Scream (HITS) screening tool.This four-item questionnaire asks respondents how frequently their partner physically hurt, insulted, threatened harm, and screamed at them during pregnancy using a five-point Likert scale ranging from "never" (coded as 1) to "frequently" (coded as 5).The total score ranges from 4 to a maximum of 20.Positive or exposure to IPV was considered if the HITS score was more significant than ten 43 .
Maternal stress was assessed based on the ten-item classic perceived stress scale (PSS) assessment instrument for one month before data collection day using a five-point Likert scale ranging from "never" (coded as 0) to "frequently" (coded as 4).The individual score on the PSS can range from 0 to a maximum of 40.Exposure to low, moderate, or high maternal stress was considered if the PSS score was 0-13, 14-26, or 27-40, respectively 44 .
Maternal anemia was assessed by measuring Hgb concentration in red blood cells by taking a finger-prick blood sample using a Hemo-Cue (Hb-201) instrument.Pregnant women with Hgb concentration < 11 g/dl were considered anemic 45.Maternal comorbidity was considered when one or more of the following medical conditions exist: cardiac disease, diabetes, thyroid disease, chronic hypertension, HIV infection, malaria, typhoid, or renal disease in the baseline assessment 46 .
Substance use was examined in pregnant women who consumed local alcohol or beer or chewed khat at least once a week during pregnancy.

Statistical analysis
The data were collected using the Open Data Kit (ODK) platforms and were exported to the R statistical programming software version 4.2.0 47 .The baseline characteristics of the women were summarised in a table with frequencies and proportions.The distribution was assessed using histograms for maternal age, gestational age, and MUAC at baseline.The median and interquartile range (IQR) for the pregnant women's age, gestational age, and MUAC were presented.The mean and standard deviation (SD) were also used to present the baseline Hgb level of women.We performed Little's missing completely at random (MCAR) test and checked the pattern of missing values.The p-value (< 0.001) indicated that the missing was not MCAR.However, the test result is insufficient to indicate whether the missing is not missing at random (NMAR) or missing at random (MAR) 48 .We then performed multiple imputations by chained equation using the "mice" package with ten imputations and 20 iterations 49 .Sensitivity analyses were performed to determine whether the MAR assumption was valid.The MAR assumption was valid, and the complete case and imputed data analysis results were comparable (Supplementary Table S1).

Model development
Individual predictors that significantly contribute to the risk of preterm birth were examined using univariable logistic regression analysis.The Likelihood Ratio Test (LRT) was used to determine the p-value for each model.Variables with a p-value for the LRT less than 0.25 were considered eligible to be included in the multivariable logistic regression analysis.The backward stepwise elimination technique with a p-value ≤ 0.15 for the LRT was fitted to build a final multivariable logistic regression model.The predictive accuracy of the final model was checked using discrimination (AUC) and calibration (calibration graphs and Hosmer-Lemeshow goodness of fit test) parameters.The Hosmer-Lemeshow goodness of fit test with a p-value greater than 0.05 indicates good calibration, which means that the probability of preterm birth estimated by the model is similar to the observed probability.An AUC of 0.5 indicates no discrimination ability, while an AUC of 1 indicates perfect discrimination.
Moreover, the added value of maternal stress in predicting preterm birth was assessed using logistic regression analysis.The model's performance, including maternal stress, was evaluated using discrimination and calibration.The AUC of this model was compared with the reduced model to evaluate the improvement in prediction performance.

Internal validation
The model was internally validated using a bootstrap technique 50 to estimate the degree of over-optimism of the final model when applied to a similar population.Internal validation was performed on the regression coefficient with a 95% confidence interval (CI) and the AUC of the model using 2,000 random bootstrap samples.The AUC difference between the bootstrap and the original full sample measured the optimism of the predictive model.

Simplified risk score development
Based on the final model's regression coefficients hierarchy, a simplified risk score was computed to provide an easily applicable prediction model.Each regression coefficient of the predictors in the final model was divided by the smallest regression coefficient, and the result was rounded to the nearest integers.The risk score performance was assessed and compared with the original regression coefficient model using the AUC.The simplified risk score is also arbitrarily classified based on the size of each interval and its potential public health relevance.
The TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist 51 was used to guide the development and validation of the prediction model and reporting (Supplementary Table S2).

Ethics declarations
The study has been reviewed, and ethical clearance was obtained from the Institutional Review Boards of Addis Ababa University, College of Health Sciences (code: 099/17/SPH).Written informed consent and parental assent were obtained from study participants.This manuscript was approved by the Faculty of Medicine and Health Sciences, University of Antwerp, Belgium, and all methods were carried out in accordance with relevant guidelines and regulations.

Result Characteristics of pregnant women
Of the 618 pregnant women, 137 were excluded from the analysis due to absence or inappropriate determination of gestational age at birth.At enrolment, the median age was 26 with an IQR of 6 years, and 38 (8.1%) were younger than 20 years (Table 1).All women (100%) were married, and 205 (42.7%) did not receive formal education.

Prediction model
Preterm birth occurred in 94 (19.5%) of the 481 women who gave birth in the BHDSS.The multivariable logistic regression analysis included variables with a p-value of less than 0.25 for the LRT in the univariable logistic regression analysis, such as MUAC, gravidity, history of abortion, anemia, comorbidity, IPV, and history of ANC for the current pregnancy.The final prediction model combined seven predictors, including MUAC, gravidity, history of abortion, comorbidity, IPV, anemia, and history of ANC for the current pregnancy (Table 2).

Model performance and validation
The AUC of the final model was 0.687 (95%CI: 0.620, 0.753).The calibration test of the model had a p-value of 0.7134, indicating good agreement between the predicted and observed probability of preterm birth (Fig. 2).
The bootstrapping technique showed that we expect low optimism when applied to newly pregnant women in a similar population.The adjusted AUC of the model was 0.689 (95% CI: 0.622, 0.755) with an AUC overoptimism coefficient of 0.002 (p-value = 0.796) (Fig. 3).

Clinical importance of maternal stress
In the univariable and multivariable analysis, the maternal stress had a p-value for the LRT of 0.031 and 0.008, respectively.The calibration plot indicates that the model, including maternal stress, had good calibration (p-value = 0.825) (Fig. 4).The AUC increased from 0.687 (95%CI: 0.620, 0.753) to 0.693 (95% CI: 0.620-0.766)with a p-value of 0.784.Therefore, the addition of maternal stress to the prediction model has not made a significant difference in the discrimination performance of the model.

Simplified risk score per individual
A MUAC less than 23 cm had the smallest regression coefficient and was weighted as 1.The total risk score ranged from 0 to 13.The simplified risk score had an AUC of 0.678 (95% CI: 0.612-0.743)and a p-value of 0.08 compared to the original regression coefficient model.Total risk score formula = (1*MUAC < 23 cm + 1*primigravida + 2*being anemic + 2*positive IPV + 3*had a history of abortion + 2* had no ANC history for the current pregnancy + 2*presence of comorbidity) (Table 3).Table 4 indicates that the risk of preterm birth increased as the simplified risk score increased.The proportion of preterm births was 244 (13.9%) with a simplified risk score of ≤ 1 and 18 (50%) with a simplified risk score of ≥ 5.
Table 5 presents results from the likelihood ratio of each risk score interval.The likelihood ratio increased as the simplified risk score for the prediction model increased.The simplified risk score between 5 and 13 increased the probability of having a preterm birth in pregnant women by 3.8 times compared to those who did not have a preterm birth.

Discussion
In the present study, one-fifth of the babies were born prematurely.We developed a prediction model for preterm birth based on the baseline characteristics of pregnant women in rural settings.The model combined MUAC, gravidity, comorbidity, ANC follow-up, history of abortion, IPV, and anemia to predict preterm birth.We developed a simplified risk score to improve the practical applicability of the prediction model in clinical practice in primary healthcare settings without requiring advanced diagnostic tests and imaging.
Previous studies developed a model for predicting preterm birth during pregnancy using ultrasound examination 26,27 and biomarker tests 28,29 .However, these procedures are limited by their expense and complexity, requiring trained healthcare professionals and specialized equipment.In LMICs, such as Ethiopia, most primary care settings lack access to these advanced laboratory and imaging procedures.Consequently, estimating the risk of preterm birth in resource-limited settings using these methods is challenging.Hence, we developed a model using easily obtained and routinely collected maternal characteristics applicable in primary care settings in Ethiopia.
Prediction models were developed for women with signs of preterm birth in tertiary care centers 35,36,38 .The potential predictors of preterm birth in hospitalized women with preterm birth symptoms may differ from those in non-hospitalized women.The healthcare system in tertiary healthcare is different, and the diagnostic procedures are more advanced and expensive.However, the current study estimated the risk of preterm birth using available characteristics of women in the primary care setting.
Prediction models for preterm birth were initially developed in comprehensive specialized hospitals in northern Ethiopia, utilizing a retrospective study design 33,34 .These studies, however, overlooked fundamental predictors recommended by WHO, such as IPV, Substance use, MUAC, history of contraceptive use, deworming, and maternal stress.The exclusive use of a specialized hospital may have limited the generalizability of findings, potentially reflecting specific patient characteristics or a more selective group seeking specialized care.In contrast, our prediction model was deployed in southern Ethiopia using a prospective study design considering the basic WHO recommendations for potential preterm birth predictors.This was crucial in minimizing recall bias and strengthening model accuracy.Additionally, our research was conducted in a rural community setting, where diagnostic methods and healthcare professionals are more limited compared to specialized hospitals.Consequently, the previously developed model may not be applicable for predicting preterm birth in primary care settings.
Schaaf et al. 53 predicted the risk of preterm birth with poor calibration and an AUC of 0.63 using a combined 13 potential predictors of preterm birth, including fetal sex and vaginal bleeding before 20 weeks of gestation.However, confirmation of fetal sex requires ultrasound examination, and vaginal bleeding before 20 weeks of pregnancy necessitates advanced diagnostic procedures.Therefore, the study is less applicable in clinical settings with scarce resources.Huang et al. 32 also developed a prediction model for preterm birth by combining stress and metabolic predictors.Maternal stress biomarkers (cortisol) and metabolites were measured in the serum samples to predict preterm birth.The prediction model yielded an optimum AUC value of 0.895.The current study  53 calculated the predictive model at two arbitrary cut-off points of the predictive probability.The incidence of preterm birth has occurred in 3.8% of pregnancies.At a predictive probability of 0.1, the sensitivity was 4.2%, and the specificity was 99.3%.At this cut-off point, the PPV was 19.4%, and the NPV was 96.3%.While in our study, we developed different cut-off points using the Youden index.The optimal predicted probability cut-off point was 0.221, with a sensitivity of 41%, a specificity of 82%, a PPV of 36%, and an NPV of 84%.The LR + value was 2.24, and the LR− value was 0.72.The simplified risk score is highly applicable and easier to use in daily clinical practice.
Our study suggests that a prediction model using the characteristics of pregnant women might be useful in identifying pregnant women at high risk of preterm birth in resource-limited settings.Women categorized as high-risk could be referred for further assessment and therapeutic intervention to prevent preterm birth.A simplified risk score would help produce relevant information for the community, policymakers, and clinical interventions to reduce neonatal morbidity and mortality rates.
The current study included highly applicable and routinely collected characteristics of pregnant women in LMICs.This can be used in daily clinical practice in primary care settings to identify high-risk pregnant women.We further used an internal validation based on a bootstrapping technique to provide unbiased estimates of the high risk for preterm birth.We developed different optimal cut-off points based on the coefficient and risk score, which helps to choose various cut-off points depending on the program goal and the availability of resources.
Our study is subject to limitations.The missingness data, mainly the history of abortion, preterm birth, and ANC for the previous pregnancy, had high missing values.However, we tried to minimize the risk of bias by using multiple imputations to develop the estimated regression coefficient of the prediction model.
When pregnant women experience risk or complication during their pregnancy, healthcare providers may intervene to induce preterm birth, affecting the prediction model and implying a computing risk.Future researchers should consider such issues using advanced analysis techniques.Potential predictors of preterm birth and complications might be different in the first and second trimesters.Developing a prediction model in each trimester thus improves the model's practical applicability and predictive capacity.Future research should consider conducting an extensive and multicentre study to improve the prediction model's generalization and external validation.

Conclusions
In this study, we developed a prediction model and a score for preterm birth risk stratification in rural Ethiopia.The model can identify pregnant women at high risk of preterm birth.Prediction of high-risk women based on individual characteristics could help to strengthen the clinical decision-making of the primary health care providers to reduce obstetric complications.It would also help produce relevant information for the community, policymakers, and clinical interventions on preventing and treating preterm birth and reducing neonatal morbidity and mortality rates.

Figure 1 .
Figure 1.Flow chart of the study design for preterm births in the Butajira Nutrition, Mental Health, and Pregnancy cohort.

Figure 2 .
Figure 2. Model performance of the prediction model for preterm birth.(A) The area under the receiver operating characteristics curve to evaluate the discrimination.The AUC suggests that the model has a 68.7% (95%CI:62.0%,75.3%)chance to correctly distinguish a high risk for preterm birth from normal pregnancy based on the characteristics of pregnant women in resource limited setting.(B) Calibration plot to evaluate the calibration of the prediction model.The visual calibration plot between the observed and prediction risk in different percentiles of the predicted values.The p-value of the calibration plot is 0.713.

Figure 3 .
Figure 3. Area under the receiver operating characteristic curve of an internal validation of the prediction model.

Figure 4 .
Figure 4. Model performance after adding maternal stress in the prediction model.(A) Area under the receiver operating characteristic curve of the final model with and without maternal stress.(B) Calibration plot to evaluate the calibration of the addition of maternal stress to the final model.The added value maternal stress was assessed through a calibration plot, revealing a good calibration (a p-value of 0.825).

Table 1 .
Baseline sociodemographic, obstetric, and clinical characteristics of study participants (n = 481).Comorbidity considered when one or more of the following medical conditions exist: cardiac disease, diabetes, thyroid disease, chronic hypertension, HIV infection, malaria, typhoid, or renal disease; ¶ Substance use: exposed to either local alcohol or beer or chewing khat at least once per week; MUAC, Mid-Upper Arm Circumference; ANC, antenatal care, IPV, intimate partner violence. #

Table 2 .
Association between predictors and preterm birth in South Ethiopia (n = 481).*p-value of LRT < 0.25 and included in multivariable analysis **p-value of LRT < 0.15 and fitted in the final model such as MUAC, gravidity, intimate partner violence, anemia, history of abortion, history of ANC, and comorbidity: #Comorbidity considered when one or more of the following medical conditions exist: cardiac disease, diabetes, thyroid disease, chronic hypertension, HIV infection, malaria, typhoid, or renal disease; (-) not included in multivariable analysis (p-value > 0.25); ANC: Antenatal Care; MUAC: Mid-Upper Arm Circumference; Ref: Reference.

Table 3 .
The simplified risk score and rounded weight to predict preterm birth.
*Regression coefficient of each predictor; **the weight of each regression coefficient divided by 0.403 and rounded to the nearest integers; # Comorbidity was considered when one or more of the following medical conditions exist: cardiac disease, diabetes, thyroid disease, chronic hypertension, HIV infection, malaria, typhoid, or renal disease; ANC: Antenatal Care; MUAC: Mid-Upper Arm Circumference; ¶ total weight: the total rounded risk score of the predictors.The probability of preterm birth = 1

Table 4 .
Risk of preterm birth per individual risk score to predict high risk for preterm birth.
aThe risk of preterm birth per individual risk score was calculated by listing the assigned total weight and counting the number of women in each leveled weight Vol.:(0123456789) Scientific Reports | (2024) 14:4845 | https://doi.org/10.1038/s41598-024-55627-z

Table 5 .
Interval likelihood ratio of risk score intervals for the prediction model.

Table 6 .
Performance assessed maternal stress using a perceived stress scale.The addition of maternal stress in the prediction model increases the AUC from 0.687 to 0.693, but the difference was insignificant compared to the original model.Schaaf et al.