Ultrasound Obstet Gynecol 2016; 48: 397–402 Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/uog.15854

External validation of IOTA simple descriptors and simple rules for classifying adnexal masses ´ ´ T. ERRASTI*, J. L. ALCAZAR*, M. A. PASCUAL†, B. GRAUPERA†, M. AUBA*, B. OLARTECOECHEA*, A. RUIZ-ZAMBRANA*, L. HERETER†, S. AJOSSA‡ and S. GUERRIERO‡ *Department of Obstetrics and Gynecology, Clinica Universidad de Navarra, University of Navarra, Pamplona, Spain; †Department of Obstetrics, Gynecology and Reproduction, Institut Dexeus, University Autonoma of Barcelona, Barcelona, Spain; ‡Department of Obstetrics and Gynecology, Policlinico Universitario Duilio Casula, University of Cagliari, Monserrato, Cagliari, Italy

K E Y W O R D S: adnexal mass; diagnosis; simple descriptors; simple rules; ultrasound

ABSTRACT Objective To assess the diagnostic performance of a three-step strategy proposed by the International Ovarian Tumor Analysis (IOTA) Group for discriminating between benign and malignant adnexal masses. Methods This was a prospective observational study, performed at two tertiary-care university hospitals, of women diagnosed with an adnexal mass on transvaginal or transabdominal ultrasound between December 2012 and December 2014. Women were scheduled for an ultrasound evaluation, which was initially performed by nonexpert examiners. The examiner had to classify the mass using ‘simple descriptors’ (first step) and, if not possible, using ‘simple rules’ (second step). For inconclusive masses, an expert examiner classified the mass according to their subjective impression (third step). Masses were managed expectantly, with serial follow-up examinations, or surgically, according to ultrasound findings and clinical symptoms. Histology was used as the reference standard. Masses that were managed expectantly with at least 1 year of follow-up were considered as benign for analytical purposes. Women with less than 1 year of follow-up were not included in the study. Results Six hundred and sixty-six women were included (median age, 41 (range, 18–81) years) of whom 514 were premenopausal and 152 were postmenopausal. Based on the three-step strategy, 362 women had surgical removal of the mass (53 malignant and 309 benign), 71 masses resolved spontaneously and 233 persisted. Four hundred and forty-eight (67.3%) of 666 masses could be classified using simple descriptors and, of the 218 that could not, 147 (67.4%) were classified using simple rules. Of the remaining 71 masses, the expert examiner classified 45 as benign, 12 as malignant and 14 as uncertain.

Overall sensitivity, specificity, positive likelihood ratio and negative likelihood ratio of the three-step strategy were 94.3%, 94.9%, 18.6 and 0.06, respectively. Conclusion The IOTA three-step strategy, based on the sequential use of simple descriptors, simple rules and expert evaluation, performs well for classifying adnexal masses as benign or malignant. Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

INTRODUCTION Accurate characterization of adnexal masses is crucial for appropriate management. Ultrasound is currently considered to be the first-line imaging technique for assessing adnexal masses1 , and several studies have shown that the examiner’s subjective impression has good diagnostic performance for characterizing adnexal masses by this modality2,3 . However, the diagnostic performance of this method depends on the examiner’s expertise4 . Unfortunately there is a limited number of expert examiners, so the majority of adnexal masses are initially evaluated by non-experts. In order to improve the diagnostic performance of non-expert examiners, many scoring systems and logistic models have been developed5 . However, many of these are complex. The International Ovarian Tumor Analysis (IOTA) Group proposed a simpler approach, based on various ultrasound features of the tumor, the so-called ‘simple rules’6 and ‘simple descriptors’, also called ‘easy instant diagnosis’7 . The simple rules have been shown to perform well when used by non-expert examiners8 , and these approaches are appealing because of their simplicity. Based on these simple approaches, IOTA proposed a clinically oriented three-step strategy. In this strategy,

´ Correspondence to: Dr J. L. Alcazar, Department of Obstetrics and Gynecology, Clinica Universidad de Navarra, Avenida Pio XII 36, 31008, Pamplona, Spain (email: [email protected]) Accepted: 23 December 2015

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

ORIGINAL PAPER

´ Alcazar et al.

398

simple descriptors, simple rules and evaluation by an expert examiner are used sequentially in order to classify adnexal masses as benign or malignant7 . This strategy is interesting since it starts by using simple approaches that could classify most adnexal masses, leaving a minority of cases to be assessed by an expert examiner. If this strategy could be shown to have a high diagnostic performance in the classification of adnexal masses, especially when examinations are carried out initially by non-expert examiners, it could lead to a more rational and efficient use of resources. However, before being introduced into clinical practice it needs to be validated externally in different clinical settings. To date, few studies have assessed the diagnostic performance of this strategy9,10 . The aim of this study was to perform an external validation of the IOTA three-step strategy for classifying adnexal masses as benign or malignant, when ultrasound is performed initially by non-expert examiners using simple descriptors and simple rules.

METHODS Study design This was a prospective observational study performed at two tertiary-care university hospitals in Spain (Clinica Universidad de Navarra, Pamplona and Institut Dexeus, Barcelona). Patients were recruited consecutively from December 2012 to December 2014. Institutional review board approval was obtained at both centers and all patients gave informed consent. STROBE guidelines for prospective studies and STARD guidelines for diagnostic studies were used11,12 .

Patients Patients were eligible if they were over 18 years of age and presented with at least one adnexal mass. They were scheduled to undergo transvaginal or transrectal (for example if virgo intacta) ultrasound examination of the adnexal mass. Exclusion criteria were: pregnancy at the time of the ultrasound evaluation, refusal to undergo transvaginal or transrectal ultrasound, surgical removal of the tumor more than 4 months after the ultrasound scan (this was an arbitrary criterion) and follow-up of less than 12 months at the time of data analysis (30th March 2015) if expectant management was chosen. Patients were managed according to their ultrasound findings and clinical symptoms. The management protocol was the same in both institutions. Women with an adnexal mass considered as malignant at any step of the assessment were referred to the gynecological oncology division for preoperative evaluation and surgical removal of the tumor (cytoreductive surgery and tumor staging). Women with an adnexal mass considered to be benign but presenting with complaints (pelvic or abdominal pain or discomfort) underwent surgery by a general gynecologist. Asymptomatic women with an

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

adnexal mass considered to be benign were offered surgery by a general gynecologist or expectant management with serial follow-up scans (twice a year). In cases of bilateral masses, the one considered as malignant or the larger mass if both were considered benign or malignant was used for analysis. Therefore, only one mass per patient was analyzed.

Ultrasound evaluation All included women underwent transvaginal or transrectal ultrasound using a Voluson E8 machine equipped with an RIC 5–9-MHz endovaginal probe (GE Healthcare Ultrasound, Milwaukee, WI, USA) in both centers. In cases with a large tumor, transabdominal ultrasound was also performed. Non-expert examiners (staff gynecologists trained in ultrasound but without special interest in gynecological ultrasound) performed the initial ultrasound examination. According to the IOTA three-step strategy, they first applied the simple descriptors (first step). Simple descriptors are based on ultrasound findings and serum CA-125 measurements (Table 1). If the mass could not be classified as benign or malignant according to simple descriptors, the same examiner applied simple rules to classify the mass (second step). Simple rules are based on five ultrasound features suggestive of malignancy and five ultrasound features suggestive of benignity (Table 1). Finally, if the mass could not be classified using simple rules, an expert examiner evaluated the mass and classified it as benign, malignant or uncertain according to their subjective impression (step three). In the case of an uncertain classification, the mass was managed as a malignant tumor and the woman was referred to the gynecological oncology division.

Reference standard For tumors that were removed surgically, definitive histological diagnosis was used as the reference standard. Malignant tumors were staged according to the International Federation of Gynaecology and Obstetrics classification13 . For analytical purposes, borderline ovarian tumors were considered as malignant; masses that were managed expectantly and therefore lacked a definitive reference standard were considered as benign if the lesion resolved spontaneously or no ovarian cancer was diagnosed after 1 year of follow-up.

Statistical analysis Continuous variables were expressed as mean ± SD and range, and categorical variables were expressed as number and percentage. Sensitivity and specificity, as well as positive and negative likelihood ratios (LR+ and LR–), and their corresponding 95% CIs were calculated for the three-step strategy. We also calculated the sensitivity and specificity for each step of the strategy. For this

Ultrasound Obstet Gynecol 2016; 48: 397–402.

Validation of IOTA descriptors and rules for adnexal masses

399

Table 1 Simple descriptors and simple rules for classifying adnexal masses as benign or malignant, as defined by the International Ovarian Tumor Analysis Group Simple descriptors

Simple rules

Benign descriptors Unilocular tumor with ground-glass echogenicity in premenopausal woman Unilocular tumor with mixed echogenicity and acoustic shadows in premenopausal woman Unilocular anechoic tumor with regular walls and largest diameter of lesion < 100 mm Unilocular tumor with regular walls Malignant descriptors Tumor with ascites and at least moderate color Doppler blood flow in postmenopausal woman Woman aged > 50 years and CA-125 > 100 IU/mL

Benign features Unilocular tumor Largest diameter of largest solid component < 7 mm Acoustic shadows Smooth multilocular tumor with largest diameter < 100 mm No intratumoral blood flow on color or power Doppler

purpose, masses that could not be classified using simple descriptors in Step 1 or using simple rules in Step 2 and those classified as uncertain in Step 3 were considered malignant. The sensitivity and specificity of each step were compared using McNemar’s test. The diagnostic performance of each center was also calculated. Statistical calculations were performed using GraphPad software (GraphPad Software, Inc., La Jolla, CA, USA).

Malignant features Irregular solid tumor Ascites At least four papillary projections Irregular multilocular solid tumor with largest diameter ≥ 100 mm Very strong intratumoral blood flow on color or power Doppler

Eligible women (n = 742) Excluded (n = 76)

Women who had surgery (n = 362) Pamplona (n = 162) Barcelona (n = 200)

Expectant management (n = 304) Pamplona (n = 53) Barcelona (n = 251)

Histology Benign (n = 309) Malignant (n = 53)

Spontaneous resolution (n = 71) Persistent mass (n = 233)

RESULTS During the study period, 742 women with an adnexal mass were evaluated. Seventy-six women were excluded for the following reasons: pregnancy at the time of ultrasound (n = 12); refusal to undergo transvaginal or transrectal ultrasound (n = 7); surgery performed > 4 months after the ultrasound scan (n = 7; all cases were benign); and follow-up of < 12 months (n = 50). Ultimately, 666 women were included (215 in Pamplona and 451 in Barcelona). Median age was 41 years (range, 18–81; interquartile range, 17 years). One hundred and fifty-two (22.8%) women were postmenopausal. Reasons for the initial ultrasound examination were pelvic pain in 166 (24.9%), menstrual disorder in 27 (4.1%), abdominal swelling or discomfort in 100 (15.0%), referral for a second opinion in 253 (38.0%) and an abnormal finding during a routine gynecological check-up in 120 (18.0%). Six hundred and forty-nine women underwent transvaginal ultrasound and 17 underwent transrectal ultrasound. Three hundred and sixty-two (54.3%) women underwent surgery following the three-step strategy (Figure 1). Of the tumors removed surgically, 309 were benign and 53 were malignant (10 were metastatic tumors to the ovary) (Table 2). Stage of primary ovarian malignant tumors was Stage I in 20 (46.5%), Stage II in four (9.3%), Stage III in 18 (41.9%) and Stage IV in one (2.3%).

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

Figure 1 Flowchart of patient selection and management of women with adnexal mass at the two centers involved in the study.

Of the 304 women managed expectantly, seventy-one (23.4%) had tumors that resolved spontaneously and 233 (76.6%) had lesions that persisted during follow-up. At the time of data analysis, no patient with an adnexal mass that had a persistent appearance had developed signs or symptoms of ovarian cancer, and all remained asymptomatic. Figure 2 shows the flow-chart of the IOTA three-step strategy. Four hundred and forty-eight (67.3%) masses could be classified using simple descriptors. One hundred and forty-seven (67.4%) of the remaining 218 masses could be classified using simple rules. Therefore, 595/666 (89.3%) masses could be classified by non-expert examiners. Seventy-one masses were evaluated subsequently by expert examiners; 45 were classified as benign, 12 as malignant and 14 as uncertain. The observed accuracies of simple descriptors, simple rules when simple descriptors were inconclusive and the expert examiner’s subjective impression when both

Ultrasound Obstet Gynecol 2016; 48: 397–402.

´ Alcazar et al.

400 Table 2 Histological diagnosis of 362 tumors removed surgically from women undergoing the three-step strategy for classifying adnexal tumors as benign or malignant Diagnosis

n (%)

Benign Endometrioma Dermoid cyst Serous/simple cyst Mucinous cyst Ovarian fibroma Cystadenofibroma Paraovarian cyst Hemorrhagic cyst Hydrosalpinx Follicular cyst Tubo-ovarian abscess Brenner tumor Retroperitoneal adenomyoma Peritoneal cyst Malignant Primary epithelial ovarian cancer Metastatic tumor to the ovary Granulosa cell tumor Borderline tumor

110 (30.4) 69 (19.1) 48 (13.3) 26 (7.2) 18 (5.0) 16 (4.4) 6 (1.7) 6 (1.7) 3 (0.8) 2 (0.6) 2 (0.6) 1 (0.3) 1 (0.3) 1 (0.3) 35 (9.7) 10 (2.8) 1 (0.3) 7 (1.9)

simple descriptors and simple rules were inconclusive are reported in Table 3. The sequential use of each step increased significantly the specificity without a significant decrease in sensitivity.

When considering only the cases with conclusive results in the first two steps, the results were as follows: for simple descriptors (n = 448), sensitivity, specificity, LR+ and LR– were 96.0% (95% CI, 80.5–99.3%), 99.5% (95% CI, 98.3–99.9%), 203.0 (95% CI, 50.8–801.1) and 0.04 (95% CI, 0.006–0.274), respectively. For simple descriptors followed by simple rules (n = 595), respective values were 94.5% (95% CI, 83.1–98.6%), 99.6% (95% CI, 98.7–99.9%), 256.1 (95% CI, 64.1–1023.5) and 0.05 (95% CI, 0.01–0.20). The diagnostic performance of the three-step strategy in each center is shown in Table 4; we did not find statistically significant differences between performance in the two centers.

DISCUSSION In the present study, we assessed the diagnostic performance of a clinically oriented three-step strategy proposed by IOTA, based on the sequential use of simple descriptors, simple rules and an examiner’s subjective impression for classifying adnexal masses as benign or malignant. This study constitutes an external validation of this approach, and we found that the strategy has a high diagnostic performance. The main strengths of our study were the large number of masses included and its prospective design. In addition, although only two centers participated in the study, the strategy performed well in both.

Masses included (n = 666) Use of simple descriptors

Masses not classified by simple descriptors (n = 218) Use of simple rules

Masses not classified by simple rules (n = 71) Use of expert examiner impression

Classified, n = 448 Benign, n = 422 RS benign, n = 421 RS malignant, n = 1 Malignant, n = 26 RS benign, n = 2 RS malignant, n = 24

Classified, n = 147 Benign, n = 118 RS benign, n = 117 RS malignant, n = 1 Malignant, n = 29 RS benign, n = 16 RS malignant, n = 13

Classified as benign (n = 45)

Classified as malignant (n = 12)

Classified as uncertain (n = 14)

RS benign, n = 44 RS malignant, n = 1

RS benign, n = 2 RS malignant, n = 10

RS benign, n = 11 RS malignant, n = 3

Figure 2 Flowchart of three-step strategy for classifying adnexal mass as benign or malignant in 666 women, showing classification and results according to reference standard (RS) for each step.

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

Ultrasound Obstet Gynecol 2016; 48: 397–402.

Validation of IOTA descriptors and rules for adnexal masses

401

Table 3 Overall diagnostic performance of three-step strategy for classifying adnexal tumors as benign or malignant using simple descriptors (SD) only, SD followed by simple rules (SR), and SD and SR followed by expert examiner’s subjective assessment (SA) Strategy SD only SD and SR SD, SR and SA

Sensitivity (%)

Specificity (%)*

LR+

LR–

98.1 (90.2–99.7) 96.2 (87.2–98.9) 94.3 (84.6–98.1)

68.7 (64.9–72.2) 87.8 (84.5–90.1) 94.9 (92.9–96.4)

3.1 (2.8–3.5) 7.9 (6.3–9.8) 18.6 (13.1–26.4)

0.03 (0.004–0.19) 0.04 (0.01–0.17) 0.06 (0.02–0.18)

Values in parentheses are 95% CI. *P < 0.05 for comparison of specificity between strategies: SD and SR vs SD only; SD, SR and SA vs SD and SR; SD, SR and SA vs SD only. LR+, positive likelihood ratio; LR–, negative likelihood ratio. Table 4 Diagnostic performance of three-step strategy for classifying adnexal tumors as benign or malignant in each center

Center Pamplona Barcelona

n

OC prevalence (%)

Sensitivity (%)

Specificity (%)

LR+

LR–

215 451

18 3

97.4 (86.8–99.5) 87.5 (63.4–96.5)

92.6 (87.8–95.6) 95.9 (93.6–97.4)

13.2 (7.8–22.3) 21.2 (13.0–34.6)

0.03 (0.004–0.19) 0.13 (0.04–0.47)

Values in parentheses are 95% CI. LR+, positive likelihood ratio; LR–, negative likelihood ratio; OC, ovarian cancer.

Among the limitations of the study is the fact that almost half of the masses were not removed surgically. Therefore, the best reference standard (histological diagnosis) was lacking in many cases. However, we consider expectant management in benign-appearing adnexal masses to be safe and acceptable practice14 . Furthermore, none of the women who were managed expectantly had been diagnosed with ovarian cancer at the time of writing. Another limitation could be the fact that ultrasound evaluation was initially performed by non-expert examiners. However, this could also be considered a strength, since we found that diagnostic performance was high, meaning that simple descriptors and simple rules used by a non-expert examiner could correctly classify many adnexal masses (as many as 89.3% in our study). In our opinion, this is clinically relevant because, in real-life clinical settings, most masses are evaluated in the first instance by non-expert examiners. The number of women who underwent transrectal ultrasound examination was too low to perform subgroup analysis on the diagnostic performance of ultrasound by this approach. Therefore our results should only be generalized to situations in which transvaginal ultrasound is used. However, it should be borne in mind that transrectal ultrasound offers similar images to those obtained on transvaginal ultrasound15 . Finally, our study design does not allow for ascertainment of whether the three-step strategy (simple descriptors followed by simple rules and then subjective assessment) is better than a two-step strategy (simple rules followed by subjective assessment). Our results are similar to those reported in the literature. In the original IOTA study, the reported sensitivity and specificity were both 92%7 . This study comprised 1938 women, and all ultrasound examinations were performed by expert examiners. 80.7% of masses could be classified using simple descriptors and/or simple rules. After this first study, two external validations have been reported9,10 .

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

Sayasneh et al.9 reported on a study performed in three UK centers involving 301 women. Ultrasound examinations were performed by Level-II examiners. As in our study, 89% of masses could be classified using simple descriptors and/or simple rules. They reported a sensitivity and specificity of 93% and 92%, respectively. Data were reported on diagnostic performance of the three-step strategy when the simple rules were non-conclusive and on subjective assessment when both simple descriptors and simple rules were non-conclusive, showing no apparent difference in diagnostic performance (sensitivity and specificity for each strategy 93% and 97%, 95% and 95%, and 96% and 83%, respectively). However, this analysis was performed considering only cases classifiable by simple descriptors (n = 167), only cases classifiable by simple descriptors and simple rules (n = 268) and for all cases (n = 301). Analyzing their data in depth and considering all 301 masses and masses that were not possible to classify as malignant with simple descriptors or simple descriptors and simple rules, the results would be as follows: sensitivity and specificity for the one-step, two-step and three-step strategies: 96% and 43%; 95% and 83%; and 93% and 92%, respectively. These figures are similar to ours, with an increase in specificity without a significant decrease in sensitivity. Testa et al.10 reported an external validation involving 18 European centers and 2403 women. All ultrasound evaluations were performed by expert examiners. Eighty percent of the masses could be classified using simple descriptors and/or simple rules. They did not report on the diagnostic performance of simple descriptors alone. When comparing the diagnostic performance of the two-step strategy (simple descriptors and simple rules, assuming tumors that could not be classified using simple rules to be malignant) and of the three-step strategy, the sensitivity and specificity reported were 95.7% and 73.6%, and 92.5% and 87.6%, respectively. In this study, the use of the three-step strategy also showed a better specificity than did the two-step strategy, with similar sensitivity.

Ultrasound Obstet Gynecol 2016; 48: 397–402.

´ Alcazar et al.

402

An interesting finding of our study, which has not been reported previously, is that the three-step strategy seems to have a similar performance in centers with a low, and those with a high, prevalence of ovarian cancer. We had three false-negative cases: one serous borderline Stage Ia tumor that was missed by simple descriptors, one serous borderline Stage Ia tumor that was missed by simple rules and a mucinous borderline Stage Ia tumor that was missed by the expert examiner. Sayasneh et al.9 had four false-negative cases using the three-step strategy: two mucinous borderline tumors, one invasive mucinous cystadenocarcinoma and another case for which the histopathology was not reported. Testa et al.10 reported 66 false-negative cases for the three-step strategy: 34 cases were borderline tumors, 13 were primary invasive ovarian cancer Stage I, seven were primary invasive ovarian cancer Stage II–IV and 12 were metastatic tumors to the ovary. These data are interesting because the majority of the false-negative cases in these three studies were borderline tumors or early-stage invasive cancers. The vast majority of advanced and metastatic ovarian cancers are detected using this sequential strategy. We believe that future research should focus on attempting to improve our ability to identify borderline tumors and early-stage invasive cancers. The role of the ADNEX model proposed by IOTA could be relevant for this task16 . The findings of our study and those published previously may have clinical relevance, since the sensitivity achieved was high with a good specificity. Perhaps the main finding of our study is that the number of women referred ultimately for expert examination was very low (71/666). This is important, since it implies that non-expert examiners can handle most adnexal masses reasonably well. However, one question remains unanswered: is a three-step strategy better than a two-step strategy? We think that further studies, such as a randomized trial, are needed to answer this question. In conclusion, our data and those from IOTA studies show that the clinically oriented three-step strategy, based on the sequential use of simple descriptors, simple rules and an expert examiner’s subjective impression, performs well in classifying adnexal masses as benign or malignant.

Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

REFERENCES 1. American College of Obstetricians and Gynecoclogists. ACOG Practice Bulletin. Management of adnexal masses. Obstet Gynecol 2007; 110: 201–214. 2. Sokalska A, Timmerman D, Testa AC, Van Holsbeke C, Lissoni AA, Leone FP, Jurkovic D, Valentin L. Diagnostic accuracy of transvaginal ultrasound examination for assigning a specific diagnosis to adnexal masses. Ultrasound Obstet Gynecol 2009; 34: 462–470. ´ 3. Alcazar JL, Guerriero S, Laparte C, Ajossa S, Ruiz-Zambrana A, Melis GB. Diagnostic performance of transvaginal gray-scale ultrasound for specific diagnosis of benign ovarian cysts in relation to menopausal status. Maturitas 2011; 68: 182–188. 4. Van Holsbeke C, Daemen A, Yazbek J, Holland TK, Bourne T, Mesens T, Lannoo L, Boes AS, Joos A, Van De Vijver A, Roggen N, de Moor B, de Jonge E, Testa AC, Valentin L, Jurkovic D, Timmerman D. Ultrasound experience substantially impacts on diagnostic performance and confidence when adnexal masses are classified using pattern recognition. Gynecol Obstet Invest 2010; 69: 160–168. 5. Kaijser J, Sayasneh A, Van Hoorde K, Ghaem-Maghami S, Bourne T, Timmerman D, Van Calster B. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update 2014; 20: 449–462. 6. Timmerman D, Testa AC, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C, Paladini D, Van Calster B, Vergote I, Van Huffel S, Valentin L. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet Gynecol 2008; 31: 681–690. 7. Ameye L, Timmerman D, Valentin L, Paladini D, Zhang J, Van Holsbeke C, Lissoni AA, Savelli L, Veldman J, Testa AC, Amant F, Van Huffel S, Bourne T. Clinically oriented three-step strategy for assessment of adnexal pathology. Ultrasound Obstet Gynecol 2012; 40: 582–591. ´ Olartecoechea B, Graupera B, Auba´ M, Ajossa S, Hereter ´ 8. Alcazar JL, Pascual MA, ´ B, Peddes C, Sedda F, Piras A, Saba L, Guerriero S. IOTA simple L, Julve R, Gaston rules for discriminating between benign and malignant adnexal masses: prospective external validation. Ultrasound Obstet Gynecol 2013; 42: 467–471. 9. Sayasneh A, Kaijser J, Preisler J, Johnson S, Stalder C, Husicka R, Guha S, Naji O, Abdallah Y, Raslan F, Drought A, Smith AA, Fotopoulou C, Ghaem-Maghami S, Van Calster B, Timmerman D, Bourne T. A multicenter prospective external validation of the diagnostic performance of IOTA simple descriptors and rules to characterize ovarian masses. Gynecol Oncol 2013; 130: 140–146. 10. Testa A, Kaijser J, Wynants L, Fischerova D, Van Holsbeke C, Franchi D, Savelli L, Epstein E, Czekierdowski A, Guerriero S, Fruscio R, Leone FP, Vergote I, Bourne T, Valentin L, Van Calster B, Timmerman D. Strategies to diagnose ovarian cancer: new evidence from phase 3 of the multicentre international IOTA study. Br J Cancer 2014; 111: 680–688. 11. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 2007; 370: 1453–1457. 12. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC; STARD Group. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Fam Pract 2004; 21: 4–10. 13. Berek JS, Crum C, Friedlander M. Cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet 2012; 119 (Suppl 2): S118–S129. ´ 14. Alcazar JL, Olartecoechea B, Guerriero S, Jurado M. Expectant management of adnexal masses in selected premenopausal women: a prospective observational study. Ultrasound Obstet Gynecol 2013; 41: 582–588. ˙ ¸ c¸i H, Yigiter ˙ The utility of transrectal ultrasound ¨ uc ¨ u¨ N, Sidar G, Is ˘ ¨ 15. Gud AB, Dunder I. in adolescents when transabdominal or transvaginal ultrasound is not feasible. J Pediatr Adolesc Gynecol 2013; 26: 265–268. 16. Van Calster B, Van Hoorde K, Valentin L, Testa AC, Fischerova D, Van Holsbeke C, Savelli L, Franchi D, Epstein E, Kaijser J, Van Belle V, Czekierdowski A, Guerriero S, Fruscio R, Lanzani C, Scala F, Bourne T, Timmerman D; International Ovarian Tumour Analysis Group. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ 2014; 349: g5920.

Ultrasound Obstet Gynecol 2016; 48: 397–402.