Advertisement

Comparison of machine learning classification techniques to predict implantation success in an IVF treatment cycle

  • Pakize Yiğit
    Correspondence
    Corresponding author.
    Affiliations
    Department of Medical Statistics and Medical Informatics, Istanbul Medipol University, Faculty of Medicine Istanbul, Turkey
    Search for articles by this author
  • Abdulbari Bener
    Affiliations
    Department of Medical Statistics and Medical Informatics, Istanbul Medipol University, Faculty of Medicine Istanbul, Turkey

    Department of Evidence for Population Health Unit, School of Epidemiology and Health Sciences, The University of Manchester Manchester, UK
    Search for articles by this author
  • Seda Karabulut
    Affiliations
    Department of Histology and Embryology, Medipol International School of Medicine, Istanbul Medipol University Istanbul, Turkey
    Search for articles by this author

      Highlights

      • Nine different models were compared
      • Ensemble models have better prediction accuracy in this study.
      • Age, transfer day and oestradiol concentration are significant predictors of IVF implantation

      Abstract

      Research question

      Which machine learning model predicts the implantation outcome better in an IVF cycle? What is the importance of each variable in predicting the implantation outcome in an IVF cycle?

      Design

      Retrospective cohort study comprising 939 transferred embryos between 2014 and 2018 in an IVF centre in Turkey with 17 selected features. The algorithms were Logistic Regression (LR), Decision Tree (DT), Naïve Bayes (NB), Random Forest (RF), Support Vector Machine (SVM), Neural Network (Nnet), Gradient Boost Decision Tree (GBDT), eXtreme Gradient Boosting (XGBoost) and Super Learner (SL). The results were evaluated with performance metrics (F1 score, specificity, accuracy and area under the receiver operating characteristic curve [AUROC]) with 10-fold cross-validation repeated ten times.

      Results

      RF and SL models achieved the highest performance and showed F1 scores of 74% and 73%, specificity of 94%, an accuracy of 89%, and AUROC of 83%. In addition, the model identified the top features as maternal age, embryo transfer day, total gonadotrophin dose and oestradiol concentration.

      Conclusions

      The present study revealed that machine learning algorithms successfully predicted implantation rates in an IVF attempt. In addition, maternal age is by far the most important predictor of IVF success when compared with other variables.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Reproductive BioMedicine Online
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Abdala A.
        • Elkhatib I.
        • Bayram A.
        • Arnanz A.
        • El-Damen A.
        • Melado L.
        • Lawrenz B.
        • Fatemi H.M.
        • De Munck N.
        Day 5 vs day 6 single euploid blastocyst frozen embryo transfers: which variables do have an impact on the clinical pregnancy rates?.
        J. Assist. Reprod. Genet. 2022; 39: 379-388https://doi.org/10.1007/s10815-021-02380-1
        • Aggarwal C.C.
        Data Classification Algorithms and Applications.
        CRC Press, Fifth Edition, Boca Raton, FL, USA2015
        • Archer K.J.
        • Kimes R.V.
        Empirical characterization of random forest variable importance measures.
        Comput. Stat. Data Anal. 2008; 52: 2249-2260https://doi.org/10.1016/j.csda.2007.08.015
        • Barnett-Itzhaki Z.
        • Elbaz M.
        • Butterman R.
        • Amar D.
        • Amitay M.
        • Racowsky C.
        • Orvieto R.
        • Hauser R.
        • Baccarelli A.A.
        • Machtinger R.
        Machine learning vs. classic statistics for the prediction of IVF outcomes.
        J. Assist. Reprod. Genet. 2020; https://doi.org/10.1007/s10815-020-01908-1
        • Blank C.
        • Wildeboer R.R.
        • DeCroo I.
        • Tilleman K.
        • Weyers B.
        • de Sutter P.
        • Mischi M.
        • Schoot B.C.
        Prediction of implantation after blastocyst transfer in in vitro fertilization: a machine-learning perspective.
        Fertil. Steril. 2019; 111: 318-326https://doi.org/10.1016/j.fertnstert.2018.10.030
        • Brás de Guimarães B.
        • Martins L.
        • Metello J.L.
        • Ferreira F.L.
        • Ferreira P.
        • Fonseca J.M.
        Application of artificial intelligence algorithms to estimate the success rate in medically assisted procreation.
        Reprod. Med. 2020; 1: 181-194https://doi.org/10.3390/reprodmed1030014
        • Breiman L.E.O.
        Random forests.
        Machine Learning. 2001; 45: 5-32
        • Broer S.L.
        • van Disseldorp J.
        • Broeze K.A.
        • Dolleman M.
        • Opmeer B.C.
        • Bossuyt P.
        • Eijkemans M.J.C.
        • Mol B.W.J.
        • Broekmans F.J.M.
        • Anderson R.A.
        • Ashrafi M.
        • Bancsi L.
        • Caroppo E.
        • Copperman A.
        • Ebner T.
        • Geva M.E.
        • Erdem M.
        • Greenblatt E.M.
        • Jayaprakasan K.
        • Fenning R.
        • Klinkert E.R.
        • Kwee J.
        • Lambalk C.B.
        • La Marca A.
        • McIlveen M.
        • Merce L.T.
        • Muttukrishna S.
        • Nelson S.M.
        • Ng H.Y.
        • Popovic-Todorovic B.
        • Smeenk J.M.J.
        • Tomás C.
        • Van der Linden P.J.Q.
        • van Rooij I.A.
        • Vladimirov I.K.
        Added value of ovarian reserve testing on patient characteristics in the prediction of ovarian response and ongoing pregnancy: an individual patient data approach.
        Hum. Reprod. Update. 2013; 19: 26-36https://doi.org/10.1093/humupd/dms041
        • Barreto N.C.N.
        • Castro G.Z.
        • Pereira R.G.
        • Pereira F.A.N.
        • Reis F.M.
        • Junior W.M.
        • Cavallo I.K.D.
        • Gomes K.B.
        Predicting in vitro fertilization success in the Brazilian public health system: a machine learning approach.
        Med. Biol. Eng. Comput. 2022; 60: 1851-1861https://doi.org/10.1007/s11517-022-02569-1
        • Cai Q.
        • Wan F.
        • Appleby D.
        • Hu L.
        • Zhang H.
        Quality of embryos transferred and progesterone levels are the most important predictors of live birth after fresh embryo transfer: a retrospective cohort study.
        J. Assist. Reprod. Genet. 2014; 31: 185-194https://doi.org/10.1007/s10815-013-0129-4
        • Chawla N.V.
        • Bowyer K.W.
        • Hall L.O.
        • Kegelmeyer W.P.
        SMOTE: Synthetic Minority Over-sampling Technique.
        J. Artif. Intell. Res. 2002; 16: 321-357https://doi.org/10.1613/jair.953
      1. Chen, T., He, T., Benesty, M., & Khotilovich, V. 2019. Package ‘xgboost’. R version, 90, 1–66.https://cran.r-project.org/web/packages/xgboost/vignettes/xgboost.pdf

        • Cimadomo D.
        • Fabozzi G.
        • Vaiarelli A.
        • Ubaldi N.
        • Ubaldi F.M.
        • Rienzi L.
        Impact of maternal age on oocyte and embryo competence.
        Front. Endocrinol. (Lausanne). 2018; 9: 327https://doi.org/10.3389/fendo.2018.00327
        • Cooper T.G.
        • Noonan E.
        • Eckardstein S.Von
        • Auger J.
        • Baker H.W.G.
        • Behre H.M.
        • Haugen T.B.
        • Kruger T.
        • Wang C.
        • Mbizvo M.T.
        • Vogelsong K.M.
        World Health Organization reference values for human semen 16, 231–245.
        Hum. Reprod. Update. 2010; 16: 231-245https://doi.org/10.1093/humupd/dmp048
        • Corani G.
        • Magli C.
        • Giusti A.
        • Gianaroli L.
        • Gambardella L.M.
        A Bayesian network model for predicting pregnancy after in vitro fertilization.
        Comput. Biol. Med. 2013; 43: 1783-1792https://doi.org/10.1016/j.compbiomed.2013.07.035
        • Curchoe C.L.
        • Malmsten J.
        • Bormann C.
        • Shafiee H.
        • Flores-Saiffe Farias A.
        • Mendizabal G.
        • Chavez-Badiola A.
        • Sigaras A.
        • Alshubbar H.
        • Chambost J.
        • Jacques C.
        • Pena C.A.
        • Drakeley A.
        • Freour T.
        • Hajirasouliha I.
        • Hickman C.F.L.
        • Elemento O.
        • Zaninovic N.
        • Rosenwaks Z.
        Predictive modeling in reproductive medicine: where will the future of artificial intelligence research take us?.
        Fertil. Steril. 2020; 114: 934-940https://doi.org/10.1016/j.fertnstert.2020.10.040
        • Dhillon R.K.
        • McLernon D.J.
        • Smith P.P.
        • Fishel S.
        • Dowell K.
        • Deeks J.J.
        • Bhattacharya S.
        • Coomarasamy A.
        Predicting the chance of live birth for women undergoing IVF: a novel pretreatment counselling tool.
        Hum. Reprod. 2016; 31: 84-92https://doi.org/10.1093/humrep/dev268
        • Elkum N.
        • Fahim M.
        • Shoukri M.
        • Al-Madouj A.
        Which patients wait longer to be seen and when? A waiting time study in the emergency department.
        East. Mediterr. Heal. J. 2009; 15: 416-424
      2. Eric Polley, Erin LeDell, Chris Kennedy, Sam Lendle, Mark van der Laan, 2019. Super Learner Prediction–R package ‘SuperLearner’.

        • Friedman J.H.
        Stochastic gradient boosting.
        Comput. Stat. Data Anal. 2002; 38: 367-378https://doi.org/10.1016/S0167-9473(01)00065-2
        • Gardner D.K.
        • Lane M.
        • Stevens J.
        • Schlenker T.
        • Schoolcraft W.B.
        Blastocyst score affects implantation and pregnancy outcome: towards a single blastocyst transfer.
        Fertil. Steril. 2000; 73: 1155-1158https://doi.org/10.1016/S0015-0282(00)00518-5
        • Goyal A.
        • Kuchana M.
        • Ayyagari K.P.R.
        Machine learning predicts live-birth occurrence before in-vitro fertilization treatment.
        Sci. Rep. 2020; 10: 1-12https://doi.org/10.1038/s41598-020-76928-z
        • Guh R.S.
        • Wu T.C.J.
        • Weng S.P.
        Integrating genetic algorithm and decision tree learning for assistance in predicting in vitro fertilization outcomes.
        Expert Syst. Appl. 2011; 38: 4437-4449https://doi.org/10.1016/j.eswa.2010.09.112
        • Güvenir H.A.
        • Misirli G.
        • Dilbaz S.
        • Ozdegirmenci O.
        • Demir B.
        • Dilbaz B.
        Estimating the chance of success in IVF treatment using a ranking algorithm.
        Med. Biol. Eng. Comput. 2015; 53: 911-920https://doi.org/10.1007/s11517-015-1299-2
        • Haas J.
        • Meriano J.
        • Bassil R.
        • Barzilay E.
        • Casper R.F.
        What is the optimal timing of embryo transfer when there are only one or two embryos at cleavage stage?.
        Gynecol. Endocrinol. 2019; 35: 665-668https://doi.org/10.1080/09513590.2019.1580259
        • Hafiz P.
        • Nematollahi M.
        • Boostani R.
        • Jahromi B.N.
        Predicting implantation outcome of in vitro fertilization and intracytoplasmic sperm injection using data mining techniques.
        Int. J. Fertil. Steril. 2017; 11: 184-190https://doi.org/10.22074/ifs.2017.4882
        • Hassan M.R.
        • Al-Insaif S.
        • Hossain M.I.
        • Kamruzzaman J.
        A machine learning approach for prediction of pregnancy outcome following IVF treatment.
        Neural Comput. Appl. 2020; 32: 2283-2297https://doi.org/10.1007/s00521-018-3693-9
        • Iliodromiti S.
        • Nelson S.M.
        Ovarian response biomarkers: physiology and performance.
        Curr. Opin. Obstet. Gynecol. 2015; https://doi.org/10.1097/GCO.0000000000000175
        • Inhorn M.C.
        • Patrizio P.
        Infertility around the globe: new thinking on gender, reproductive technologies and global movements in the 21st century.
        Hum. Reprod. Update. 2014; 21: 411-426https://doi.org/10.1093/humupd/dmv016
        • Jurisica I.
        • Mylopoulos J.
        • Glasgow J.
        • Shapiro H.
        • Casper R.F.
        Case-based reasoning in IVF: prediction and knowledge mining.
        Artif. Intell. Med. 1998; 12: 1-24https://doi.org/10.1016/S0933-3657(97)00037-7
        • Kaufman S.
        • Rosset S.
        • Perlich C.
        • Stitelman O.
        Leakage in data mining: formulation, detection, and avoidance.
        ACM Trans. Knowl. Discov. Data. 2012; 6: 1-21https://doi.org/10.1145/2382577.2382579
        • Kaufmann S.J.
        • Eastaugh J.L.
        • Snowden S.
        • Smye S.W.
        • Sharma V.
        The application of neural networks in predicting the outcome of in-vitro fertilization.
        Hum. Reprod. 1997; 12: 1454-1457https://doi.org/10.1093/humrep/12.7.1454
        • Kim I.-C.
        • Jung Y.-G.
        Using Bayesian networks to analyze medical data.
        in: Perner P. Rosenfeld A. Proceedings of the Third International conference of Machine Learning and Data Mining (MLDM). Leipzig. Springer-Verlag, 2003: 317-327
      3. Kothandaraman, R., Andavar, S., Raja Soosaimarian, P. R., 2021. Dynamic model for assisted reproductive technology outcome prediction 64, 1–14.

      4. Kuhn, M., 2020. caret: Classification and Regression Training.https://CRAN.R-project.org/package=caret R package version 6.0-86.

        • La Marca A.
        • Sunkara S.K.
        Individualization of controlled ovarian stimulation in IVF using ovarian reserve markers: from theory to practice.
        Hum. Reprod. Update. 2014; 20: 124-140https://doi.org/10.1093/humupd/dmt037
        • Lai I.
        • Neal M.
        • Gervais N.
        • Amin S.
        • Taerk E.
        • Faghih M.
        Transfers of lower quality embryos based on morphological appearance result in appreciable live birth rates: a Canadian center's experience.
        F S Reports. 2020; 1: 264-269https://doi.org/10.1016/j.xfre.2020.09.003
        • Liaw A.
        • Wiener W.
        Package ‘randomForest’.
        Ann. Oper. Res. 2018; https://doi.org/10.1023/A:1010933404324
        • Lundberg S.M.
        • Lee S.-I.
        A Unified Approach to Interpreting Model Predictions.
        in: 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA2017https://doi.org/10.1016/j.ophtha.2018.11.016
        • McLernon D.J.
        • Steyerberg E.W.
        • Te Velde E.R.
        • Lee A.J.
        • Bhattacharya S.
        Predicting the chances of a live birth after one or more complete cycles of in vitro fertilisation: population based study of linked cycle data from 113 873 women.
        BMJ. 2016; 355: i5735https://doi.org/10.1136/bmj.i5735
        • Mehrjerd A.
        • Rezaei H.
        • Eslami S.
        • Ratna M.B.
        • Khadem Ghaebi N.
        Internal validation and comparison of predictive models to determine success rate of infertility treatments: a retrospective study of 2485 cycles.
        Sci. Rep. 2022; 12: 1-11https://doi.org/10.1038/s41598-022-10902-9
        • Mencar C.
        • Mantero M.
        • Tarsia P.
        • Carpagnano G.E.
        • Barbaro M.P.F.
        • Lacedonia D.
        Application of machine learning to predict obstructive sleep apnea syndrome severity.
        Health Informatics J. 2019; 26: 298-317https://doi.org/10.1177/1460458218824725
        • Milewski R.
        • Milewska A.J.
        • Wiesak T.
        • Morgan A.
        Comparison of artificial neural networks and logistic regression analysis in pregnancy prediction using the in vitro fertilization treatment.
        Stud. Logic, Gramm. Rhetor. 2013; 35: 39-48https://doi.org/10.2478/slgr-2013-0033
        • Mirroshandel S.A.
        • Ghasemian F.
        • Monji-Azad S.
        Applying data mining techniques for increasing implantation rate by selecting best sperms for intra-cytoplasmic sperm injection treatment.
        Comput. Methods Programs Biomed. 2016; 137: 215-229https://doi.org/10.1016/j.cmpb.2016.09.013
        • Nachtigall R.D.
        International disparities in access to infertility services.
        Fertil. Steril. 2006; 85: 871-875https://doi.org/10.1016/J.FERTNSTERT.2005.08.066
        • Ombelet W.
        • Cooke I.
        • Dyer S.
        • Serour G.
        • Devroey P.
        Infertility and the provision of infertility medical services in developing countries.
        Hum. Reprod. Update. 2008; 14: 605-621https://doi.org/10.1093/humupd/dmn042
        • Omobolaji R.
        • Elmusrati M.
        • Sawazaki I.
        • Paulo L.
        • Haglund C.
        • Coletta R.D.
        • Mäkitie A.A.
        • Salo T.
        • Almangush A.
        • Leivo I.
        Comparison of supervised machine learning classification techniques in prediction of locoregional recurrences in early oral tongue cancer.
        Int. J. Med. Inform. 2020; 136104068https://doi.org/10.1016/j.ijmedinf.2019.104068
        • Qiu J.
        • Li P.
        • Dong M.
        • Xin X.
        • Tan J.
        Personalized prediction of live birth prior to the first in vitro fertilization treatment: a machine learning method.
        J. Transl. Med. 2019; 17: 1-8https://doi.org/10.1186/s12967-019-2062-5
        • Quinlan J.R.
        J. Ross Quinlan_C4.5_ Programs for Machine Learning.pdf.
        Morgan Kaufmann, 1993https://doi.org/10.1007/BF00993309
        • Raef B.
        • Ferdousi R.
        A review of machine learning approaches in assisted reproductive technologies.
        Acta Inform. Medica. 2019; 27: 205-211https://doi.org/10.5455/aim.2019.27.205-211
        • Raef B.
        • Maleki M.
        • Ferdousi R.
        Computational prediction of implantation outcome after embryo transfer.
        Health Informatics J. 2020; 26: 1810-1826https://doi.org/10.1177/1460458219892138
        • Ranjini K.
        • Suruliandi A.
        • Raja S.P.
        An ensemble of heterogeneous incremental classifiers for assisted reproductive technology outcome prediction.
        IEEE Trans. Comput. Soc. Syst. 2020; 8: 557-567https://doi.org/10.1109/TCSS.2020.3032640
        • Ratna M.B.
        • Bhattacharya S.
        • Abdulrahim B.
        • McLernon D.J.
        A systematic review of the quality of clinical prediction models in in vitro fertilisation.
        Hum. Reprod. 2020; 35: 100-116https://doi.org/10.1093/humrep/dez258
        • Sacha C.R.
        • Dimitriadis I.
        • Christou G.
        • Souter I.
        • Bormann C.L.
        The effect of day 2 versus day 3 embryo transfer on early pregnancy outcomes in women with a low yield of fertilized oocytes.
        J. Assist. Reprod. Genet. 2018; 35: 879-884https://doi.org/10.1007/s10815-018-1157-x
        • Spiessens C.
        • De Neubourg D.
        • Peeraer K.
        • D'Hooghe T.
        • Chen F.
        • Debrock S.
        Selecting the embryo with the highest implantation potential using a data mining based prediction model.
        Reprod. Biol. Endocrinol. 2016; 14: 1-13https://doi.org/10.1186/s12958-016-0145-1
        • Staessen C.
        • Camus M.
        • Bollen N.
        • Devroey P.
        • Van Steirteghem A.C.
        The relationship between embryo quality and the occurrence of multiple pregnancies.
        Fertil. Steril. 1992; 57: 626-630https://doi.org/10.1016/s0015-0282(16)54911-5
        • Steyerberg E.W.
        Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating.
        2nd edn. Springer Series Statistics for Biology and Health, 2019
        • Tu J.V.
        Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes.
        J. Clin. Epidemiol. 1996; 49: 1225-1231
        • Ubaldi F.M.
        • Cimadomo D.
        • Vaiarelli A.
        • Fabozzi G.
        • Venturella R.
        • Maggiulli R.
        • Mazzilli R.
        • Ferrero S.
        • Palagiano A.
        • Rienzi L.
        Advanced maternal age in IVF: still a challenge? The present and the future of its treatment.
        Front. Endocrinol. (Lausanne). 2019; https://doi.org/10.3389/fendo.2019.00094
        • Uyar A.
        • Bener A.
        • Ciray H.N.
        Predictive modeling of implantation outcome in an in vitro fertilization setting.
        Med. Decis. Mak. 2015; 35: 714-725https://doi.org/10.1177/0272989×14535984
        • van Buuren S.
        • Groothuis-Oudshoorn K.
        mice: Multivariate Imputation by Chained Equations in R.
        J. Stat. Softw. 2011; 45: 1-67https://doi.org/10.18637/jss.v045.i03
        • Van der Laan M.J.
        • Rose S.
        Targeted Learning (Preface).
        Springer Series in Statistics, 2011https://doi.org/10.1007/978-1-4419-9782-1
        • Van Der Laan M.J.
        • Polley E.C.
        • Hubbard A.E.
        Super Learner.
        Statist. App. Genet. Molec. Biol. 2007; 6https://doi.org/10.2202/1544-6115.1309
        • Vander Borght M.
        • Wyns C.
        Fertility and infertility: definition and epidemiology.
        Clin. Biochem. 2018; 62: 2-10https://doi.org/10.1016/j.clinbiochem.2018.03.012
        • Wald M.
        • Sparks A.E.T.
        • Sandlow J.
        • Van-Voorhis B.
        • Syrop C.H.
        • Niederberger C.S.
        Computational models for prediction of IVF/ICSI outcomes with surgically retrieved spermatozoa.
        Reprod. Biomed. Online. 2005; 11: 325-331https://doi.org/10.1016/S1472-6483(10)60840-1
        • Zegers-Hochschild F.
        • Adamson G.D.
        • Dyer S.
        • Racowsky C.
        • de Mouzon J.
        • Sokol R.
        • Rienzi L.
        • Sunde A.
        • Schmidt L.
        • Cooke I.D.
        • Simpson J.L.
        • van der Poel S.
        The International Glossary on Infertility and Fertility Care, 2017.
        Fertil. Steril. 2017; 108 (Epub 2017 Jul 29PMID: 28760517): 393-406https://doi.org/10.1016/j.fertnstert.2017.06.005
        • Zhao Y.Y.
        • Yu Y.
        • Zhang X.W.
        Overall blastocyst quality, trophectoderm grade, and inner cell mass grade predict pregnancy outcome in euploid blastocyst transfer cycles.
        Chin. Med. J. (Engl.). 2018; 131: 1261-1267https://doi.org/10.4103/0366-6999.232808

      Biography

      Pakize Yiğit graduated from Istanbul University with a Quantitative Methods PhD (2015) and Biostatistics PhD (2021). She has worked in the Department of Medical Statistics and Bioinformatics at Medipol University School of Medicine in Turkey since 2015. Her main research interests are multivariate statistics, multi-criteria decision-making methodologies and statistical learning techniques.
      Key message
      Tree-based ensemble models (Random Forest and eXtreme Gradient Boosting) have better prediction accuracy compared with the other algorithms in this study. Maternal age is by far the most essential predictor of IVF success when comparing other studied variables. Transfer day, oestradiol concentration and total gonadotropin dose are other significant predictors of IVF implantation.