Advertisement

Inter-centre reliability in embryo grading across several IVF clinics is limited: implications for embryo selection

  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    ,
    Author Footnotes
    # These authors contributed equally to this work.
    Danilo Cimadomo
    Correspondence
    Corresponding author.
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    # These authors contributed equally to this work.
    Affiliations
    GeneraLife IVF, Clinica Valle Giulia, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    ,
    Author Footnotes
    # These authors contributed equally to this work.
    Laura Sosa Fernandez
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    # These authors contributed equally to this work.
    Affiliations
    Embryos Fertility Center, Battipaglia Salerno, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Daria Soscia
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    GeneraLife IVF, Clinica Valle Giulia, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Gemma Fabozzi
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    GeneraLife IVF, Clinica Valle Giulia, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Francesca Benini
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    GeneraLife IVF, Demetra Center Florence, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Amalia Cesana
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    Humanitas Clinical and Research Center–IRCCS, Rozzano, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Maria Beatrice Dal Canto
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    Biogenesi Reproductive Medicine Center, Istituti Clinici Zucchi, Monza, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Roberta Maggiulli
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    GeneraLife IVF, Clinica Valle Giulia, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Saverio Muzzì
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    ICSI ROMA, Clinica Villa Mafalda, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Catello Scarica
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    European Hospital, Center for Reproductive Medicine, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Laura Rienzi
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    GeneraLife IVF, Clinica Valle Giulia, Rome, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Lucia De Santis
    Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    Affiliations
    Centro Scienze Natalità, Dept Ob/Gyn, IRCCS San Raffaele Scientific Institute, Milan, Italy
    Search for articles by this author
  • Author Footnotes
    9 On behalf of SIERR (Italian Society of Embryology, Reproduction and Research)
    # These authors contributed equally to this work.
Published:October 06, 2021DOI:https://doi.org/10.1016/j.rbmo.2021.09.022

      Abstract

      Research question

      What is the intra- and inter-centre reliability in embryo grading performed according to the Istanbul Consensus across several IVF clinics?

      Design

      Forty Day 3 embryos and 40 blastocysts were photographed on three focal planes. Senior and junior embryologists from 65 clinics were invited to grade them according to the Istanbul Consensus (Study Phase I). All participants then attended interactive training where a panel of experts graded the same embryos (Study Phase II). Finally, a second set of pictures was sent to both embryologists and experts for a blinded evaluation (Study Phase III). Intra-centre reliability was reported for Study Phase I as Cohen's kappa between senior and junior embryologists; inter-centre reliability was instead calculated between senior/junior embryologists and experts in Study Phase I versus III to outline improvements after training (i.e. upgrade of Cohen's kappa category according to Landis and Koch).

      Results

      Thirty-six embryologists from 18 centres participated (28% participation rate). The intra-centre reliability was (i) substantial (0.63) for blastomere symmetry (range –0.02 to 1.0), (ii) substantial (0.72) for fragmentation (range 0.29–1.0), (iii) substantial (0.66) for blastocyst expansion (range 0.19–1.0), (iv) moderate (0.59) for inner cell mass quality (range 0.07–0.92), (v) moderate (0.56) for trophectoderm quality (range 0.01–0.97). The inter-centre reliability showed an overall improvement from Study Phase I to III, from fair (0.21–0.4) to moderate (0.41–0.6) for all parameters under analysis, except for blastomere fragmentation among senior embryologists, which was already moderate before training.

      Conclusions

      Intra-centre reliability was generally moderate/substantial, while inter-centre reliability was just fair. The interactive training improved it to moderate, hence this workflow was deemed helpful. The establishment of external quality assessment services (e.g. UK NEQAS) and the avant-garde of artificial intelligence might further improve the reliability of this key practice for embryo selection.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Adolfsson E.
        • Andershed A.N.
        Morphology vs morphokinetics: a retrospective comparison of inter-observer and intra-observer agreement between embryologists on blastocysts with known implantation outcome.
        JBRA Assist. Reprod. 2018; 22: 228-237
        • ALPHA Scientists in Reproductive Medicine and ESHRE Special Interest Group Embryology
        Istanbul consensus workshop on embryo assessment: proceedings of an expert meeting.
        Reprod. Biomed. Online. 2011; 22: 632-646
        • ALPHA Scientists in Reproductive Medicine and ESHRE Special Interest Group Embryology
        The Istanbul consensus workshop on embryo assessment: proceedings of an expert meeting.
        Hum. Reprod. 2011; 26: 1270-1283
        • Apter S.
        • Ebner T.
        • Freour T.
        • Guns Y.
        • Kovacic B.
        • Le Clef N.
        • Marques M.
        • Meseguer M.
        • Montjean D.
        • Sfontouris I.
        • Sturmey R.
        • Coticchio G.
        Eshre Working group on Time-lapse technology: Good practice recommendations for the use of time-lapse technology.
        Hum. Reprod. Open. 2020; (2020, hoaa008)
        • Arce J.C.
        • Ziebe S.
        • Lundin K.
        • Janssens R.
        • Helmgaard L.
        • Sorensen P.
        Interobserver agreement and intraobserver reproducibility of embryo quality assessments.
        Hum. Reprod. 2006; 21: 2141-2148
        • Armstrong S.
        • Bhide P.
        • Jordan V.
        • Pacey A.
        • Farquhar C.
        Time-lapse systems for embryo incubation and assessment in assisted reproduction.
        Cochrane Database Syst. Rev. 2018; 5CD011320
        • Balaban B.
        • Yakin K.
        • Urman B.
        Randomized comparison of two different blastocyst grading systems.
        Fertil. Steril. 2006; 85: 559-563
        • Baxter Bendus A.E.
        • Mayer J.F.
        • Shipley S.K.
        • Catherino W.H.
        Interobserver and intraobserver variation in day 3 embryo grading.
        Fertil. Steril. 2006; 86: 1608-1615
        • Capalbo A.
        • Rienzi L.
        • Cimadomo D.
        • Maggiulli R.
        • Elliott T.
        • Wright G.
        • Nagy Z.P.
        • Ubaldi F.M.
        Correlation between standard blastocyst morphology, euploidy and implantation: an observational study in two centers involving 956 screened blastocysts.
        Hum. Reprod. 2014; 29: 1173-1181
        • Capalbo A.
        • Ubaldi F.M.
        • Cimadomo D.
        • Maggiulli R.
        • Patassini C.
        • Dusi L.
        • Sanges F.
        • Buffo L.
        • Venturella R.
        • Rienzi L.
        Consistent and reproducible outcomes of blastocyst biopsy and aneuploidy screening across different biopsy practitioners: a multicentre study involving 2586 embryo biopsies.
        Hum. Reprod. 2016; 31: 199-208
        • Castilla J.A.
        • Ruiz De Assin R.
        • Gonzalvo M.C.
        • Clavero A.
        • Ramirez J.P.
        • Vergara F.
        • Martinez L.
        External quality control for embryology laboratories.
        Reprod. Biomed. Online. 2010; 20: 68-74
        • Cimadomo D.
        • Capalbo A.
        • Levi-Setti P.E.
        • Soscia D.
        • Orlando G.
        • Albani E.
        • Parini V.
        • Stoppa M.
        • Dovere L.
        • Tacconi L.
        • Ievoli E.
        • Maggiulli R.
        • Ubaldi F.M.
        • Rienzi L.
        Associations of blastocyst features, trophectoderm biopsy and other laboratory practice with post-warming behavior and implantation.
        Hum. Reprod. 2018; 33 (1992–2001)
        • Cimadomo D.
        • Craciunas L.
        • Vermeulen N.
        • Vomstein K.
        • Toth B.
        Definition, diagnostic and therapeutic options in recurrent implantation failure: an international survey of clinicians and embryologists.
        Hum. Reprod. 2021; 36: 305-317
        • Cimadomo D.
        • Rienzi L.
        • Romanelli V.
        • Alviggi E.
        • Levi-Setti P.E.
        • Albani E.
        • Dusi L.
        • Papini L.
        • Livi C.
        • Benini F.
        • Smeraldi A.
        • Patassini C.
        • Ubaldi F.M.
        • Capalbo A.
        Inconclusive chromosomal assessment after blastocyst biopsy: prevalence, causative factors and outcomes after re-biopsy and re-vitrification. A multicenter experience.
        Hum. Reprod. 2018; 33: 1839-1846
        • Cimadomo D.
        • Soscia D.
        • Vaiarelli A.
        • Maggiulli R.
        • Capalbo A.
        • Ubaldi F.M.
        • Rienzi L.
        Looking past the appearance: a comprehensive description of the clinical contribution of poor-quality blastocysts to increase live birth rates during cycles with aneuploidy testing.
        Hum. Reprod. 2019; 34: 1206-1214
        • Coticchio G.
        • Ezoe K.
        • Lagalla C.
        • Shimazaki K.
        • Ohata K.
        • Ninomiya M.
        • Wakabayashi N.
        • Okimura T.
        • Uchiyama K.
        • Kato K.
        • Borini A.
        Perturbations of morphogenesis at the compaction stage affect blastocyst implantation and live birth rates.
        Hum. Reprod. 2021; 36: 918-928
        • Coticchio G.
        • Lagalla C.
        • Sturmey R.
        • Pennetta F.
        • Borini A.
        The enigmatic morula: mechanisms of development, cell fate determination, self-correction and implications for ART.
        Hum. Reprod. Update. 2019; 25: 422-438
        • De Placido G.
        • Wilding M.
        • Strina I.
        • Alviggi E.
        • Alviggi C.
        • Mollo A.
        • Varicchio M.T.
        • Tolino A.
        • Schiattarella C.
        • Dale B.
        High outcome predictability after IVF using a combined score for zygote and embryo morphology and growth rate.
        Hum. Reprod. 2002; 17: 2402-2409
        • De Los Santos M.J.
        • Apter S.
        • Coticchio G.
        • Debrock S.
        • Lundin K.
        • Plancha C.E.
        • Prados F.
        • Rienzi L.
        • Verheyen G.
        • Woodward B.
        • Vermeulen N.
        • ESHRE Guideline Group on Good Practice in IVF Labs
        Revised guidelines for good practice in IVF laboratories (2015).
        Hum. Reprod. 2016; 31: 685-686
        • Kokkali G.
        • Coticchio G.
        • Bronet F.
        • Celebi C.
        • Cimadomo D.
        • Goossens V.
        • Liss J.
        • Nunes S.
        • Sfontouris I.
        • Vermeulen N.
        • Zakharova E.
        • De Rycke M.
        • ESHRE PGT Consortium and SIG-Embryology Biopsy Working Group
        ESHRE PGT Consortium and SIG Embryology good practice recommendations for polar body and embryo biopsy for PGT.
        Hum. Reprod. Open. 2020; (2020, hoaa020)
        • Fabozzi G.
        • Alteri A.
        • Rega E.
        • Starita M.F.
        • Piscitelli C.
        • Giannini P.
        • Colicchia A.
        Morphological assessment on day 4 and its prognostic power in selecting viable embryos for transfer.
        Zygote. 2016; 24: 477-484
        • Fabozzi G.
        • Cimadomo D.
        • Maggiulli R.
        • Vaiarelli A.
        • Ubaldi F.M.
        • Rienzi L.
        Which key performance indicators are most effective in evaluating and managing an in vitro fertilization laboratory?.
        Fertil. Steril. 2020; 114: 9-15
        • Feil D.
        • Henshaw R.C.
        • Lane M.
        Day 4 embryo selection is equal to Day 5 using a new embryo scoring system validated in single embryo transfers.
        Hum. Reprod. 2008; 23: 1505-1510
        • Gardner D.K.
        • Schoolcraft W.B.
        Culture and transfer of human blastocysts.
        Curr. Opin. Obstet. Gynecol. 1999; 11: 307-311
        • Hammond E.R.
        • Foong A.K.M.
        • Rosli N.
        • Morbeck D.E.
        Should we freeze it? Agreement on fate of borderline blastocysts is poor and does not improve with a modified blastocyst grading system.
        Hum. Reprod. 2020; 35: 1045-1053
        • Ivec M.
        • Kovacic B.
        • Vlaisavljevic V.
        Prediction of human blastocyst development from morulas with delayed and/or incomplete compaction.
        Fertil. Steril. 2011; 96 (e2): 1473-1478
        • Khosravi P.
        • Kazemi E.
        • Zhan Q.
        • Malmsten J.E.
        • Toschi M.
        • Zisimopoulos P.
        • Sigaras A.
        • Lavery S.
        • Cooper L.A.D.
        • Hickman C.
        • Meseguer M.
        • Rosenwaks Z.
        • Elemento O.
        • Zaninovic N.
        • Hajirasouliha I.
        Deep learning enables robust assessment and selection of human blastocysts after in vitro fertilization.
        NPJ Digital Medicine. 2019; 2: 21
        • La Marca A.
        • Dal Canto M.
        • Buccheri M.
        • Valerio M.
        • Mignini Renzini M.
        • Rodriguez A.
        • Vassena R.
        A novel transnational fresh oocyte donation (TOD) program based on transport of frozen sperm and embryos.
        Hum. Reprod. 2019; 34: 285-290
        • Lagalla C.
        • Coticchio G.
        • Sciajno R.
        • Tarozzi N.
        • Zaca C.
        • Borini A.
        Alternative patterns of partial embryo compaction: prevalence, morphokinetic history and possible implications.
        Reprod. Biomed. Online. 2020; 40: 347-354
        • Landis J.R.
        • Koch G.G.
        An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers.
        Biometrics. 1977; 33: 363-374
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Machtinger R.
        • Racowsky C.
        Morphological systems of human embryo assessment and clinical evidence.
        Reprod. Biomed. Online. 2013; 26: 210-221
        • Maggiulli R.
        • Giancani A.
        • Cimadomo D.
        • Ubaldi F.M.
        • Rienzi L.
        Human blastocyst biopsy and vitrification.
        J. Vis. Exp. 2019; (Jul 26): 149
        • Martinez-Granados L.
        • Serrano M.
        • Gonzalez-Utor A.
        • Ortiz N.
        • Badajoz V.
        • Lopez-Regalado M.l.
        • Boada M.
        • Castilla J.A.
        • Special Interest Group in Quality of Asebir (Society for The Study of Reproductive Biology)
        Reliability and agreement on embryo assessment: 5 years of an external quality control programme.
        Reprod. Biomed. Online. 2018; 36: 259-268
        • Martinez-Granados L.
        • Serrano M.
        • Gonzalez-Utor A.
        • Ortiz N.
        • Badajoz V.
        • Olaya E.
        • Prados N.
        • Boada M.
        • Castilla J.A.
        • Special Interest Group in Quality of Asebir (Spanish Society for The Study of Reproductive Biology)
        Inter-laboratory agreement on embryo classification and clinical decision: conventional morphological assessment vs. time lapse.
        PLoS One. 2017; 12e0183328
        • Morbeck D.E.
        Blastocyst culture in the era of PGS and FreezeAlls: is a ‘C’ a failing grade?.
        Hum. Reprod. Open. 2017; (2017, hox017)
        • Paternot G.
        • Wetzels A.M.
        • Thonon F.
        • Vansteenbrugge A.
        • Willemen D.
        • Devroe J.
        • Debrock S.
        • D'hooghe T.M.
        • Spiessens C.
        Intra- and interobserver analysis in the morphological assessment of early stage embryos during an IVF procedure: a multicentre study.
        Reprod. Biol. Endocrinol. 2011; 9: 127
        • Pribenszky C.
        • Nilselid A.M.
        • Montag M.
        Time-lapse culture with morphokinetic embryo selection improves pregnancy and live birth chances and reduces early pregnancy loss: a meta-analysis.
        Reprod. Biomed. Online. 2017; 35: 511-520
        • Racowsky C.
        • Ohno-Machado L.
        • Kim J.
        • Biggers J.D.
        Is there an advantage in scoring early embryos on more than one day?.
        Hum. Reprod. 2009; 24: 2104-2113
        • Rienzi L.
        • Cimadomo D.
        • Delgado A.
        • Minasi M.G.
        • Fabozzi G.
        • Gallego R.D.
        • Stoppa M.
        • Bellver J.
        • Giancani A.
        • Esbert M.
        • Capalbo A.
        • Remohi J.
        • Greco E.
        • Ubaldi F.M.
        • Meseguer M.
        Time of morulation and trophectoderm quality are predictors of a live birth after euploid blastocyst transfer: a multicenter study.
        Fertil. Steril. 2019; 112 (e1): 1080-1093
        • Rienzi L.
        • Ubaldi F.
        • Iacobelli M.
        • Romano S.
        • Minasi M.G.
        • Ferrero S.
        • Sapienza F.
        • Baroni E.
        • Greco E.
        Significance of morphological attributes of the early embryo.
        Reprod. Biomed. Online. 2005; 10: 669-681
        • Shear M.A.
        • Vaughan D.A.
        • Modest A.M.
        • Seidler E.A.
        • Leung A.Q.
        • Hacker M.R.
        • Sakkas D.
        • Penzias A.S.
        Blasts from the past: is morphology useful in PGT-A tested and untested frozen embryo transfers?.
        Reprod. Biomed. Online. 2020; 41: 981-989
        • Storr A.
        • Venetis C.A.
        • Cooke S.
        • Kilani S.
        • Ledger W.
        Inter-observer and intra-observer agreement between embryologists during selection of a single Day 5 embryo for transfer: a multicenter study.
        Hum. Reprod. 2017; 32: 307-314
        • Sundvall L.
        • Ingerslev H.J.
        • Breth Knudsen U.
        • Kirkegaard K.
        Inter- and intra-observer variability of time-lapse annotations.
        Hum. Reprod. 2013; 28: 3215-3221
        • Tao J.
        • Tamis R.
        • Fink K.
        Pregnancies achieved after transferring frozen morula/compact stage embryos.
        Fertil. Steril. 2001; 75: 629-631
        • Tran D.
        • Cooke S.
        • Illingworth P.J.
        • Gardner D.K.
        Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer.
        Hum. Reprod. 2019; 34: 1011-1018

      Biography

      Danilo Cimadomo, MSc, PhD, is a molecular biologist, Science and Research Manager of GeneraLife IVF, member of the Executive Committee of SIERR, and Basic Science Officer of the ESHRE Special Interest Group Implantation and Early Pregnancy. He has authored over 80 peer-reviewed manuscripts concerning embryology, embryo selection and PGT.
      Key message
      Embryo grading suffers from very low inter-centre reliability and, although interactive training platforms are useful to improve it, efficient services and tools to standardize this essential feature for embryo selection are eagerly awaited in the future.