INTERPRETING ULTRASOUND ELASTOGRAPHY: IMAGE REGISTRATION OF BREAST CANCER ULTRASOUND ELASTOGRAPHY TO HISTOPATHOLOGY IMAGES Brian Chuang1, Andriy Myronenko2, Ruth English3, Alison Noble1 1

2

Institute of Biomedical Engineering, University of Oxford Department of Science and Engineering, Oregon Health & Science University 3 Oxford Radcliffe Hospital NHS Trust could be found in the B-mode image (left) that correlates to the stiff area within the red square (right) in the elastography image.

ABSTRACT Ultrasound elastography is a promising technique for the detection of breast cancer. Despite being proven to be a useful method in clinical studies, no studies have looked at directly correlating information in in-vivo ultrasound elastography images with histopathology images (regarded as the gold standard) using image registration. In this paper, expert knowledge from clinicians is utilised as constraints to register 2D elastography and histopathology images based on corresponding features identified by clinicians such as tumours and fibrous structures. The recently proposed coherent point drift (CPD) algorithm by Myronenko and Song [11] was applied to align the corresponding feature points and the thin-plate splines method of Bookstein [1] is then used to warp the images. The registered images were then overlaid. It was found that in elastography images, the stiffness of malignant tumours tend to extend beyond the tumour boundaries identified in the histopathology images, and there were many stiff areas indicated in the elastography images where no corresponding features could be identified in the histopathology images. The study thus provides some new insight into the relationship of elastography and histopathology as well as suggests further work is needed to better understand how to interpret patterns in elastography images. Index Terms— Elastography, registration, Breast cancer

Ultrasound

histopathology

1. INTRODUCTION Ultrasound elastography is a new diagnostic technique which exploits the difference in stiffness between cancerous and benign tissues and has the potential to improve breast cancer diagnosis. Results from clinical studies have so far supported the use of ultrasound elastography as a valuable addition to current screening methods [4, 5, 15]. However, the interpretation of ultrasound elastography images is by no means straightforward, and in “reallife scanning”, there are many factors that lead to artefacts and provide potentially misleading information to the diagnosis. This can be seen in figure 1 where the area within the red circles correspond to the tumour where there is a hypoechoic region in the B-mode image (left) and is indicated in the elastography image (right) as a red/stiff patch. However, no corresponding features

978-1-4244-4126-6/10/$25.00 ©2010 IEEE

181

Fig. 1. (Left) B-mode image (Right) with elastography overlaid where it is stiffer towards the red end of the colour spectrum. In order to understand the information that elastography images provide, the images need to be studied alongside with histopathology images that represent the ground truth. To achieve this goal, the images need to be registered before any comparisons can be made. The main challenge in registering histopathology images with images from other modalities is the complex tissue deformations introduced during the preparation of the histopathology slides. Existing literature on the registration of histopathology and in vivo images are mainly of MRI images of brains and prostates due to the availability of anatomical structures that can be readily identified in MRI images [2, 6, 8, 9, 12, 13]. There is a limited literature on the registration of histopathology with ultrasound images [10, 14] and as far as we are aware, none exists for registering ultrasound elastography images to histopathology images. One could try and apply the methods developed for registering MRI images to histopathology images, but this would be difficult due to the lack of anatomical structural details in ultrasound elastography images . In this paper, the image registration task is considered as a point-set registration problem where corresponding features are identified manually in the histopathology and elastography images with guidance from clinicians and the recently proposed coherent point drift (CPD) algorithm by Myronenko and Song [11] was used

ISBI 2010

Fig. 2. a) Original histopathology image. The continuous black line is the tumour boundary highlighted based on advice from clinicians and the blue dotted line represent area of dutal carcinoma in situ (DCIS). The outer boundaries are colour-coded where blue is superficial, black is deep, yellow is superior and red is inferior. b) Original elastography image. The white line is the tumour boundary highlighted by clinicians. c) Registered elastography image. The white line is the tumour boundary highlighted by clinicians. d) Fusion image of a) and c). In d, the arrow point towards stiffness in elastography extending beyond the tumour boundary in the histopathology image and the circled area shows a stiff area in elastography where no corresponding features can be found. to perform the registration. The primary purpose of doing this alignment was to gain insight into the information that ultrasound elastography provide. 2. METHODS The images used for this work were selected from a clinical study which used the assisted freehand ultrasound (AFUSON) system [7] to acquire B-mode and elastography images. The acquisition angle was chosen such that ultrasound slices were approximately aligned with histopathology slides. The histopathology slides were prepared from the preserved wide local excision specimens and were digitised and analysed. The ultrasound elastography image from each patient was paired up with a histology image that corresponds best to the elastography image by the clinicians. Having discussed with the clinicians, it was found that since the tumours and other anatomical structures are stiffer than the surrounding fatty tissues, they tend to keep their shape and are not so much affected by deformations. As a result, boundaries of tumours and fibres were selected as features to be aligned and the task was considered as a non-rigid point-set registration problem.

182

Segmentation of features in the images was followed by the guidance from the radiologist and was performed manually by highlighting the boundaries in Microsoft Paint and using the Wacom Cintiq 20WSX interactive pen-display system. The recently proposed coherent point drift (CPD) algorithm by Myronenko and Song [11] was selected for the registration task due to its robustness and efficiency. The non-rigid method proposed is particularly well suited for tackling the problem of complex deformations introduced to the histopathology due to it being able to allow for local alignments. In addition, it has never been applied to this registration problem before and does not require equal number of points in the reference and floating point set. The CPD registration code used for this work was the implementation in MATLAB® by Andriy Myronenko. http://www.bme.ogi.edu/~myron/matlab/cpd/. The algorithm considers the alignment of the floating point set onto the point set as a probability density estimation problem using Gaussian mixtures models (GMM). The floating point set is considered as GMM centroids and is transformed by a set of transformation parameters to fit onto the reference point set (considered as data points) using an expectation-maximisation (EM) algorithm to

Fig. 3. a) Original histopathology image b) Original elastography image c) Registered elastography image d) Fusion image of a) and c). Tumour boundaries identified by clinicians are delineated in the images by non-circular black lines in the histopathology images (a) and white lines in the elastography images (b and c). In d, the arrow point towards stiffness in elastography extending beyond the tumour boundary in the histopathology image maximise the likelihood. In our case, the floating point set was chosen to be the histopathology point set and the reference set was the elastography point set. The EM algorithm was used to estimate the correspondence probability between all points in the two point sets, and parameters were updated using the correspondence probability (see [11] for detail). Once the non-rigid transformation parameters were obtained, the thin-plate splines (TPS) method was then used to deform the image grid and spline interpolation was used to deform image pixels onto the new grid. All the image processing was performed using MATLAB®. In order to facilitate the comparison between the registered images, Paint.Net® was used as a visualisation tool to overlap the registered elastography images onto the histopathology images. The level of transparency of the images was adjusted to aid comparison of images.

figure, deformation of the elastography image can clearly be seen from the unregistered elastography image (b) and the registered elastography image (c) where the outline of the features used are delineated by either black (a) or white lines (b and c). Both fusion images (figure 2d and 3d) show visually good registration results in aligning the corresponding features and in deforming the elastography images. In both cases, the registered elastography images clearly indicate the area within the tumour boundary in the histopathology image to be much stiffer than the surrounding tissues. However, it can be noticed that in both cases, the stiff regions indicated by the registered elastography images extend into the surrounding tissues beyond the tumour boundaries in the histopathology images. In addition, there are regions in the registered elastography image where no corresponding features could be related in the histopathology image.

3. RESULTS

4. CONCLUSION

Figures 2 and 3 show the results of CPD non-rigid registration performed on 2 sets of patient images where the diagnoses are grade 3 invasive pleomorphic lobular carcinoma and grade 3 invasive ductal carcinoma for figure 2 and 3 respectively. In each

In this work, the aim was to develop a better understanding of the information provided in ultrasound elastography images by registering and comparing elastography images to histopathology images using the CPD algorithm by Myronenko and Song [11].

183

Expert knowledge of the clinicians were used to manually segmentate the images and used as constraints to the registration task. To our knowledge, this is the first study in the literature that performs the registration of ultrasound elastography and histopathology images. Areas within the boundaries of malignant tumours in the histopathology images identified by the clinicians were found to be stiff on the registered elastography image which suggests good correlation between malignant tumours in the histopathology image and the stiff areas in the elastography image. The interesting thing that was observed was that in all cases, stiff regions in the elastography images extended beyond the boundary of the tumours marked by the clinicians on the histopathology images. This observation is in agreement to the finding by Garra et al. [4] where possible explanations are that the tumours extend beyond what can be seen on the macroscopic level or the stiffness of the tumour extends into surrounding tissues. This would agree with the clinical finding that cancers tend to feel larger than they are. However, several stiff areas indicated in the elastography image showed no corresponding cancerous areas in the histopathology image which could be due to boundary effects or artefacts; but with the help of information provided by the B-mode image, these would not be considered to be cancerous. This reiterates the conclusion drawn by many studies that elastography provides important additional information in the diagnosis of breast cancer, but would be less useful alone. Since there was limited information on the microscopic details such as the cell types in the histopathology image, no solid conclusion can be drawn from the observation where stiffness in the elastography images extend beyond the tumour boundaries in the histopathology images. Overall, the results provide useful insights to the information in elastography images and this work serve as an important step towards understanding such information properly, which is crucial if ultrasound elastography is to be used as a diagnostic tool for breast cancer detection in the future.

5. REFERENCES [1] Bookstein, F.L., “Principal warps: Thin-plate splines and the decomposition of deformations”, IEEE T-PAMI, 11(6), 1989 [2] Breen, M.S. et al., “Three dimensional comparison of interventional MR radiofrequency ablation images with tissue response”, 6th International Conference on Medical Imaging Computing and Computer-Assisted Intervention, 2879:24-31, 2003 [3] Adams R.F. et al. “A pilot study to evaluate assisted freehand elasticity ultrasound in sizing early breast cancer: developing the rules governing elasticity image interpretation” Royal College of Radiologists Breast Group Annual Scientific Meeting in Edinburgh, Nov 2006 [4] Garra, B.S. et al., “Elastography of breast lesions: Initial clinical results”, Radiology, 202(1):79-86, 1997 [5] Itoh, A. et al., “Breast disease: Clinical application of US elastography for diagnosis”, Radiology, 239(2):341-350, 2006 [6] Jacobs, M.A. et al., “Registration and warping of magnetic resonance images to histological sections”, Medical Physics, 26(8):1568-1578, 1999

184

[7] Kadour, M.J., and Noble, J.A. “Assisted-freehand ultrasound elasticity imaging”, IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 56(1), January 2009 [8] Kim, B. et al.,” Mutual information for automated unwarping of rat brain autoradiographs”, Neuroimage, 5(1):31-40, 1997 [9] Meyer, C.R. et al., “A methodology for registration of a histological slide and in vivo MRI volume based on optimizing mutual information”, Molecular Imaging, 5(1):16-23, 2006 [10] Moskalik, A.P. et al., “3D registration of ultrasound with histology in the prostate”, IEEE Ultrasonics Symposium: 13971400, 1997 [11] Myronenko A. and Song X. “Point-set registration: coherent point drift.”, technical report arxiv:0905.2635v1, 2009 Coherent point drift (CPD) MATLAB® code: http://www.bme.ogi.edu/~myron/matlab/cpd/ [12] Park, H. et al., “Registration methodology for histological sections and in vivo imaging of human prostate”, Academic Radiology, 15(8):1027-1039, 2008 [13] Pitiot, A. et al., “Piecewise affine registration of biological images for volume reconstruction”, 2nd International Workshop on Biomedical Image Registration, 10:465-483, 2003 [14] Taylor, L.S. et al., “Three-dimensional registration of prostate images from histology and ultrasound”, Ultrasound in Medicine and Biology, 30(2):161-168, 2004 [15] Zhi, H. et al., “Comparison of ultrasound elastography, mammography, and sonography in the diagnosis of solid breast lesions”, Journal of Ultrasound in Medicine, 26(6):807-815, 2007