Prediction of clinical pregnancy after frozen embryo transfer based on ultrasound radiomics: an analysis based on the optimal periendometrial zone

Abstract

Background

To investigate the optimal periendometrial zone (PEZ) in ultrasound (US) images and assess the performance of ultrasound radiomics in predicting the outcome of frozen embryo transfer (FET).

Methods

This prospective study had 422 female participants (training set: n = 358, external validation set: n = 64). We delineated the region of interest (ROI) of the endometrium (EN) from ultrasound images of the median sagittal surface of the uteri of patients. We determined the ROIs of PEZ on US images by automatically expanding 2.0, 4.0, 6.0, and 8.0 mm radii surrounding the EN. We determined the radiomics characteristics based on the ROIs of the endometrium and PEZ, then compared the different sizes of PEZ to determine the optimal PEZ. We constructed models of the EN and optimal PEZ using six machine learning algorithms. We developed a combined model using the radiomics characteristics of EN and the optimal PEZ. We evaluated the performance of the three models using the area under the curve (AUC).

Results

The optimal PEZ was 4.0 mm with a maximum AUC of 0.715 (95% confidence interval (CI): 0.581 – 0.833) in the external validation set. The combined radiomics model (endometrium and PEZ_4.0 mm) yielded the best predictive performance with AUC = 0.853 (95% CI: 0.811 – 0.890) for the training set and AUC = 0.809 (95% CI: 0.696 – 0.909) for the external validation set.

Conclusions

PEZ_4.0 mm could be the optimal area for predicting clinical pregnancy after FET. An US-based radiomics model that combines EN and PEZ_4.0 mm demonstrated strong potential in helping clinicians predict FET outcomes more accurately, thereby supporting informed decision-making before treatment.

Introduction

Since the advent of assisted reproductive technology, clinicians and embryologists have attempted to develop methods to improve pregnancy rates. However, the success rate remains dissatisfactory [1]. Endometrial receptivity (ER) and embryo quality are two important factors that determine the success of a transplantation [2]. As a simple and noninvasive examination method, ultrasound is widely used in reproductive medicine. Researchers have attempted to determine an ultrasound indicator that can predict the outcome of embryo transfer. Some studies have reported that indicators such as endometrial thickness (EMT), volume, blood supply, and peristaltic wave affect ER [3,4,5]. However, evidence from other studies confirm that these indicators are not directly related to the implantation rate [6,7,8]. All ultrasound indicators exhibited limited effectiveness in the assessment of ER. In addition, ER markers assessed using endometrial biopsy and hysteroscopy were described to exhibit “poor ability” [8] and were invasive. Therefore, developing a non-invasive alternative method for evaluating ER is necessary.

Radiomics, a recently developed computational method, can be used to extract extensive imaging data from medical images that cannot be identified by the naked eye. It has been used in various classification tasks [9]. The junctional zone (JZ) between the endometrium and the myometrium can undergo periodic changes under the influence of estrogen. Recent studies have shown that the peristalsis of the endometrium is closely related to this phenomenon; this can provide important information about the pregnancy-related microenvironment [10, 11]. We assumed that the intersection of the EN and the JZ can contain more comprehensive information and obtain better diagnostic capabilities. Reportedly, the average thickness of the JZ is from 5 to 8 mm, and the normal average maximum thickness is 8 mm on magnetic resonance imaging (MRI). The pregnancy rate reduces with an increase in the thickness [12]. However, MRI is an expensive and time-consuming process, and the outline of the JZ in the image obtained is neither linear or homogenous [13]. The image is not usually very clear, particularly on ultrasound. In a previous study where we applied US-based radiomics to predict pregnancy outcomes after frozen embryo transfer (FET), the region of interest (ROI) of JZ was obtained by encompassing the hypoechoic band surrounding the visible endometrium on ultrasound images as fully as possible [14]. However, in clinical practice with later models, we found that the ROI of JZ often exhibited poor visibility in many patients' ultrasound images, which limited the clinical applicability of the prediction model we developed. Hence, we attempted to observe the optimal periendometrial zone (PEZ), which is closely associated with pregnancy, as an alternative to the JZ.

We attempted to identify the optimal PEZ using machine learning (ML) algorithms along with ultrasound images of the endometrium and PEZ to predict the ER before FET pregnancy.

Method

Study population

This prospective study included 358 women who received frozen embryo transfer (FET) at the First Affiliated Hospital of Anhui Medical University (Center 1), from August 2023 to October 2024. Additionally, we included 64 women from Hefei maternal and Child Health Hospital (Center 2) as an external testing set. The inclusion criteria were as follows: women under the age of 40 who underwent FET of one high-quality blastocyst and had a clearly visible endometrium on ultrasound examination. The embryos with scores ≥ 3AA、3AB 、3BA、3BB(day 5) or ≥ 4AA、4AB、4BA、4BB(day 6) were considered as high quality embryo blastocyst [15]. The exclusion criteria were as follows: women with congenital uterine malformations, submucosal fibroids, endometrial polyps, uterine adhesions, or uterine adenomyosis and endometrial thickness (EMT) < 8 mm or > 14 mm (Fig. 1). We recorded the history of each patient, including their age, infertility causes and body mass index (BMI). Following this, we performed a transvaginal ultrasound (TVS) evaluation to determine the exclusion criteria. The day before transplantation, we performed TVS to measure the EMT, endometrial patter, the type of Colour Doppler, with the mid-sagittal plane of the endometrium being preserved for digital imaging and communication data. The parameters of Colour Doppler are as follows: Color frequency: 5.0, Gain: 46, Wall filter: 90 Hz, Depth: 8.0. According to the Applebaum classification criteria, Type 1 is characterized by the presence of blood flow in the endometrial area near the uterine cavity. Type 2 is defined by blood flow in half of the endometrial area, and type 3 is indicated by blood flow detected solely in the subendometrial area [16]. We conducted a follow-up to assess embryo survival in patients at 5 weeks as a clinical pregnancy outcome. We compared age, EMT, and other factors with pregnancy outcomes.

Fig. 1

The flow chart of this study. ART assisted reproductive technology; EMT endometrial thickness; FET frozen embryo transfer; TVS transvaginal ultrasound

This study was conducted in accordance with the principles of the Declaration of Helsinki and approved by the Ethics Review Committee of the First Affiliated Hospital of Anhui Medical University (Approval No. PJ2023-07–11). All participants signed the relevant forms to provide informed consent.

Transfer protocol

We administered hormone replacement therapy in cycles for the treatment. The therapy commenced on the third day of the menstrual cycle, with the oral intake of 2 mg of estradiol valerate tablets (twice daily) for 10 days. We performed TVS monitoring to measure the EMT, with a threshold set at greater than 8 mm. Subsequently, we administered intramuscular progesterone injections (60 mg, once daily) and dydrogesterone (10 mg, twice daily) orally. Following a 5-day progesterone conversion period, we performed the FET procedure.

Operating methods

All enrolled women underwent TVS within 24 h before FET. The two-dimensional ultrasound scan clearly displayed an image of the standard mid-sagittal section of the endometrium, and we retrieved and stored the image. We recorded the EMT and endometrial pattern. We used color Doppler ultrasound to evaluate endometrial blood flow typing. All ultrasound examinations were conducted by two doctors with nearly 6 years of experience in reproductive ultrasound.

Endometrial radiomic feature extraction and model establishment

Ultrasound physician A, with 6 years of experience in reproductive ultrasound, who did not have access to patient information, manually delineated the region of interest (ROI) encompassing the endometrium on the sagittal section of the standard endometrial slice. Subsequently, using the open-source software tool PyRadiomics, she determined radiomics features, such as original images, square images, and wavelet filtered images, from the ROI. After half a month, ultrasound physician A and another ultrasound physician, with nearly 10 years of experience in reproductive ultrasound, used a blinded approach to delineate ROIs on randomly selected images of 30 patients and evaluated the intra-class and inter-class correlation coefficients (ICCs).

The steps adopted for processing radiomics features were as follows. First, we retained features with an ICCs > 0.75 [17]. Second, we performed a univariate rank-sum test on the features and discarded those with a p-value > 0.05. Following this, we conducted a Spearman correlation analysis on the features and Spearman’s correlation analysis with r ≥ 0.6 was used to eliminate the redundancy [18]. Next, we performed elastic-logistic analysis of the radiomics features. Finally, we used the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm to optimize model parameters through ten-fold cross-validation and select radiomics features with non-zero coefficients from the training set. We used multiple ML algorithms, including logistic regression (LR), support vector machine (SVM), random forest classifier (RF), decision tree classifier (DT), k-nearest neighbor classifier (KNN), and Back Propagation Neural Network (BPNN), to model and analyze the merged radiomics features. We evaluated the performance of the endometrial model in predicting ER using AUC, accuracy, sensitivity, and specificity. The model that achieved the best AUC was selected as the radiomics model for the endometrium from the training set and external validation set.

Radiomic feature extraction and model establishment for periendometrial zone

At the image level, first, we resampled the ultrasound images using the B-spline interpolation algorithm to align images with different spatial resolutions, ensuring that the voxel spacing was standardized to 1 × 1 mm². Next, we applied grayscale normalization to standardize the intensity levels of grayscale images from different sources or imaging devices, ensuring consistent brightness and contrast across images. At the feature level, all radiomic features in the training set were standardized using Z-scores. The parameters (i.e., mean and standard deviation) calculated from the training set were strictly applied to standardize the validation set, ensuring consistent normalization across different cohort data. We automatically expanded the ROI boundaries outward by 2, 4, 6, and 8 mm using the contour of the endometrium, creating corresponding size annular regions around the endometrium. We normalized the images and use PyRadiomics to extract features such as the original image, squared image, and wavelet-filtered image. We retained features with an ICCs > 0.75. We performed univariate rank-sum tests to analyze the significance of radiomic features, using a threshold of p < 0.05. Following this, we performed Spearman correlation analysis to identify redundant features in the 2, 4, 6, and 8 mm regions around the endometrium, defining 0.6 as the redundancy thresholds, with one of them randomly retained to avoid redundancy. Next, we performed elastic-logistic analysis on the 2, 4, 6, and 8 mm regions around the endometrium separately and evaluated the model performance using AUC curves. We used the LASSO algorithm to optimize model parameters through ten-fold cross-validation and select radiomic features with non-zero coefficients in the training set. We used various ML algorithms to model and analyze the radiomic features. We used AUC, accuracy, sensitivity, and specificity to evaluate the predictive performance of the optimal PEZ model for ER. We selected the model with the greatest AUC from the training and external validation sets and named it the radiomic model for PEZ.

Construction of a combined model

We combined the radiomics features of endometrium and PEZ to perform Spearman correlation analysis. Features with a correlation threshold > 0.6 were considered redundant, with one of them randomly retained to avoid redundancy. We then conducted elastic-logistic analysis, which yielded 15 radiomics features. We used multiple ML algorithms to model the combined radiomics features obtained from joint LASSO. We named the model with the greatest AUC in the training and external validation set the combined radiomics model. The flow chart of radiomics research is shown in Fig. 2.

Fig. 2

The radiomics flow chart of the study. PEZ periendometrial zone; ROI Regions-of- interest; EN endometrium; LR logistic regression; SVM support vector machine; RF random forest classifier; DT decision tree classifier; KNN k-nearest neighbor classifier; BPNN Back Propagation Neural Network

Explanation of the optimal model

We used the SHapley Additive exPlanations (SHAP) method to explain and visualize the feature importance of the best-performing model, which we used to solve the “black-box” issue [19]. SHAP summary plots showed the global interpretability. The SHAP values, which helped measure the impact of individual features on the model's predictions, are shown on the x-axis for each sample, whereas the y-axis shows each feature. The positive values on the graph indicate an increased likelihood of the predicted outcome, whereas negative values indicate a decrease. Each data point on the plot corresponded to an individual sample and indicated the respective SHAP value, with the color of the point indicating the magnitude of the feature value.

Statistical analysis

We used SPSS 25.0 and Python 2.7 for statistical analyses. We used mean ± standard deviation to present quantitative data with normal distribution and median ± interquartile interval to present quantitative data with non-normal distribution. We used numbers and percentages to indicate the classification variables. We conducted univariate analysis using the Mann–Whitney U test, Chi-square test, and Student’s t-test. Statistical significance was set at p < 0.05. We performed the DeLong test to form the area under the receiver operating characteristic (ROC) of the models and calculate the AUC to evaluate the diagnostic performances.

Result

Baseline characteristics

In our study, the training set had 209 clinical pregnancies (58.4%) and 149 non-clinical pregnancies (41.6%). The external validation set had 36 clinical pregnancies (56.3%) and 28 non-clinical pregnancies (43.7%). The baseline characteristics of the participants in the study are presented in Table 1.

Table 1 Clinical characteristics of patients in the training and external validation set

Confirmation of the optimal periendometrial zone (PEZ)

In the comparison among the PEZ at 2, 4, 6, and 8 mm, the 4 mm region was found to be optimal (Table 2). Using multiple machine learning algorithms, the BPNN model outperformed the others, achieving an AUC of 0.744 for the PEZ_4.0 mm group in the testing set and 0.715 in the external validation set (Table 3).

Table 2 Performance of different sizes for PEZ in the training and external validation set

Table 3 Performance of the six ML Models in the training and external validation set

Comparison of radiomics prediction models

As depicted in Table 4, the combined radiomics model (AUC: 0.853, 95% CI: 0.811 – 0.890) outperformed the endometrium model (AUC: 0.770, 95% CI: 0.716 – 0.815) and the PEZ model (AUC: 0.744, 95% CI: 0.691 – 0.794) in the training set. Furthermore, in the external validation set, the combined radiomics model (AUC: 0.809, 95% CI: 0.696 – 0.909) outperformed the endometrium model (AUC: 0.715, 95% CI: 0.581 – 0.833) and the PEZ model (AUC: 0.732, 95% CI: 0.581 – 0.833). The ROC and DCA curves of the three models are compared in Fig. 3.

Table 4 Performance of three radiomics model in the training and external validation set

Fig. 3

ROC and DCA of three models in the training and external validation set. a ROC of three models in the training set; b ROC of three models in the external validation set; (c) DCA of three models in the training set. d DCA of three models in the external validation set. DCA decision curve analysis; ROC receiver operating characteristic; EN endometrium; PEZ periendometrial zone

Analysis of feature importance for the optimal model

As illustrated in Fig. 4, we used the SHAP summary plots to sort the importance of the features, with the foremost feature displayed at the top of the plot.

Fig. 4

SHAP summary plot shows the importance of all features. Each feature on the y axis and the SHAP value on the x axis. The SHAP value means the contribution of each feature to the model prediction, with the foremost feature displaying at the top of the image

Discussion

In this study, we analyzed the performance of PEZ of different dimensions using ultrasound images in predicting pregnancy outcomes after FET. We demonstrated that the ROI of a 4.0 mm PEZ was optimal for predicting pregnancy outcome. We integrated the ultrasound image features of the endometrium and PEZ on the mid-sagittal plane to demonstrate that this approach offered better predictive potential than those based solely on assessments of the endometrial region or PEZ. We developed and validated a non-invasive, personalized ultrasound radiomics model for predicting pregnancy outcomes following FET.

Radiomics uses high-throughput data obtained by converting radiographic images into quantitative features for classification and analysis. It is used extensively in the medical domain [20]. The transitional region located between the endometrium and the myometrium is known as the JZ. It is visualized on ultrasound scans as a hypoechoic region encircling the endometrium. The thickness of the JZ demonstrates cyclic variations throughout the menstrual cycle. In the absence of pregnancy, the JZ causes endometrial peristalsis and contributes to the regulation of multiple reproductive processes, sperm transportation, and embryo implantation [21,22,23]. Its close relationship with pregnancy renders it a fundamental component for radiomics analysis. However, the JZ cannot be viewed clearly in ultrasonic images, and the binding zone appears uneven and non-linear, which makes their observation challenging. Based on previous studies, the upper limit of the normal thickness of JZ is 8 mm [12]. In this study, the maximum area surrounding the endometrium that we investigated was 8 mm, with intervals of 2 mm. Using ultrasound radiomics, we made a preliminary attempt to explore the optimal PEZ within the 2.0, 4.0, 6.0, and 8.0 mm ranges surrounding the endometrium, which is associated with pregnancy outcomes after FET. This is the first investigation of the optimal PEZ. We found that the prediction performance of PEZ_4.0 mm was better than that of other areas, suggesting that PEZ_4.0 mm may contain the most extensive microenvironment information. Endometrium-based ultrasound radiomics has been applied for outcome prediction after FET, with an AUC of 0.825 [24]. In our study, we identified radiomics features of both the endometrium and the optimal PEZ. In the training group, our combined model achieved an AUC of 0.853, outperforming models using either EN or PEZ alone. This integration enhances prediction accuracy for pregnancy outcomes after FET, providing a more comprehensive assessment of ER. By better identifying patients with higher or lower risks of successful implantation, the model may help optimize treatment strategies, such as adjusting the timing of embryo transfer or exploring alternative interventions for patients with lower predicted success rates. Ultimately, this approach has the potential to improve patient outcomes and reduce the emotional and financial burdens of failed transfers. Besides, of the six ML algorithms applied, BPNN model achieved the highest performance. BPNN, as a powerful algorithm that models complex nonlinear relationships by mimicking biological propagation behaviors, has distinct advantages in handling complex nonlinearity, automatic feature learning, generalization ability, robustness, and scalability for large datasets. These strengths often make it outperform traditional ML algorithms in many tasks [25]. This might explain why the BPNN model outperformed the other models in our study. To explain the findings of the combined model, we used SHAP summary plots to showcase the primary predictive factors for pregnancy outcomes after FET. This enhanced the transparency and interpretability of the model's predictions.

However, our study had several limitations. First, the ROI of the endometrium was manually drawn, which may have introduced subjective bias. Therefore, further research is needed using automated segmentation methods. Second, we determined the PEZ by amplifying the actual size of the region based on a computer code. However, owing to the complexity of the code, our data were non-continuous. Based on findings from previous studies, we only preliminarily compared the PEZ within the 2.0, 4.0, 6.0, and 8.0 mm regions. Other regions should be investigated further. Lastly, all participants in our study were under 40 and had no uterine abnormalities, which may limit the generalizability of our findings. To enhance the external validity, we plan to refine our clinical model and expand its validation to a larger, more diverse population across different healthcare settings. This will help assess the model's applicability and ensure its relevance to a broader patient demographic.

For the model to be successfully integrated into clinical practice, several barriers must be addressed. First, clinicians will need training to accurately interpret and use the model's predictions, ensuring they understand its functionality and can apply its insights to daily decision-making. Second, standardizing imaging protocols across institutions is crucial to ensure consistent and reliable model performance, making it applicable in various clinical settings. Despite these challenges, the potential benefits of this model in improving clinical decision-making and patient outcomes are substantial. We believe that with further validation and addressing these barriers, this model could become an integral part of personalized treatment strategies for FET, ultimately improving pregnancy success rates and reducing the emotional and financial burdens associated with unsuccessful embryo transfers.

Conclusions

We aimed to explore the performance of the PEZ in predicting the outcome of FET and determine the optimal size of the PEZ, which was found to be 4.0 mm. Our findings provide novel insights for future research on this topic. By establishing an ultrasound radiomics model based on information obtained from the EN and PEZ, we can accurately predict pregnancy outcomes after FET. By successfully applying this integrated model in clinical practice, patient outcomes could improve, while also reducing the emotional and financial burdens of FET failure.