Automatic Segmentation of Kidneys from Non-Contrast CT Images Using Efficient Belief Propagation

a

Jianfei Liua, Marius George Lingurarub, Shijun Wanga, Ronald M. Summersa Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD USA 20892-1182; b Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Washington, DC, USA 20010 ABSTRACT

CT colonography (CTC) can increase the chance of detecting high-risk lesions not only within the colon but anywhere in the abdomen with a low cost. Extracolonic findings such as calculi and masses are frequently found in the kidneys on CTC. Accurate kidney segmentation is an important step to detect extracolonic findings in the kidneys. However, noncontrast CTC images make the task of kidney segmentation substantially challenging because the intensity values of kidney parenchyma are similar to those of adjacent structures. In this paper, we present a fully automatic kidney segmentation algorithm to support extracolonic diagnosis from CTC data. It is built upon three major contributions: 1) localize kidney search regions by exploiting the segmented liver and spleen as well as body symmetry; 2) construct a probabilistic shape prior handling the issue of kidney touching other organs; 3) employ efficient belief propagation on the shape prior to extract the kidneys. We evaluated the accuracy of our algorithm on five non-contrast CTC datasets with manual kidney segmentation as the ground-truth. The Dice volume overlaps were 88%/89%, the root-mean-squared errors were 3.4 mm/2.8 mm, and the average surface distances were 2.1 mm/1.9 mm for the left/right kidney respectively. We also validated the robustness on 27 additional CTC cases, and 23 datasets were successfully segmented. In four problematic cases, the segmentation of the left kidney failed due to problems with the spleen segmentation. The results demonstrated that the proposed algorithm could automatically and accurately segment kidneys from CTC images, given the prior correct segmentation of the liver and spleen. Keywords: Kidney segmentation, extracolonic finding, CT colonography, belief propagation, Markov random field

1. INTRODUCTION Kidney segmentation is a critical step in the automated computer-aided diagnosis of the abdomen from CT data. An important application is the detection of extracolonic findings from CT colonography (CTC) images because 8.6% of the patients undergoing CTC exams were found to have unexpected extracolonic findings1. However, the detection of extracolonic findings is not the primary screening purpose of the CTC protocols, which address the detection of colon cancer. As a result, the kidneys are poorly imaged in the non-contrast CTC images. Fig. 1 compares kidney imaging on a non-contrast CTC image and a contrast-enhanced CT image. In Fig. 1(a), the kidneys have similar intensity distributions to their surrounding tissues, muscles, and adjacent organs. Accurate segmentation of the kidneys becomes even more difficult if the kidney parenchyma touches its neighboring structures such as muscles, which easily leads to oversegmentation. In contrast, kidneys are significantly enhanced in Fig. 1(b) and their intensity values are higher than the neighboring objects because of intravenous contrast material. Distinctive kidney boundaries simplify the task of kidney segmentation and this desirable property makes it tractable to precisely segment kidneys on contrast-enhanced CT images. In recent years a large body of research has been devoted to kidney segmentation, the majority focusing on contrastenhanced CT images. To constrain the kidney variability among individuals, probabilistic atlases are frequently used in kidney segmentation, such as Ali’s work5. The probabilistic atlas is integrated into a Markov random field represented as an energy function. Kidneys are segmented by using the graph cut algorithm13 to minimize the energy. Yuksel6 extended this work to segment the kidney on dynamic contrast enhanced MRI images by introducing an additional probabilistic model to describe kidney intensity distribution. Another improvement over Ali’s work is to replace a parametric atlas with a non-parametric iterative model5 in the Markov random field. Khalifa9 reformulates the model as a statistical speed function and imports it into the level-set framework to extract kidneys, instead of directly segmenting kidneys using the Medical Imaging 2013: Computer-Aided Diagnosis, edited by Carol L. Novak, Stephen Aylward, Proc. of SPIE Vol. 8670, 867005 · © 2013 SPIE · CCC code: 1605-7422/13/$18 · doi: 10.1117/12.2007738

Proc. of SPIE Vol. 8670 867005-1

graph cut method. Lin7 chooses spine positions as landmarks to localize kidney ranges and search for elliptic structures as the possible kidney candidates. Adaptive region growing algorithm is used to segment kidneys on the candidate regions. Many other researchers fit the kidney segmentation into the multi-organ segmentation scheme8,12. In comparison with single organ segmentation, this framework exploits the spatial relationships among different organs. This potentially reduces over-segmentation through the constraints from other organs. Recently, machine learning techniques11 were applied to kidney segmentation by introducing more comprehensive prior knowledge into the segmentation framework. This strategy employs many training examples to support the process to acquire accurate prior information. Nevertheless, almost all these methods take advantage of contrast-enhanced CT images, and it is infeasible to directly apply them to our CTC images because kidneys are imaged with substantial variance between non-contrast and contrast CT images, as illustrated in Fig. 1.

(a) Non-Contrast CT (b) Contrast CT Fig. 1: Comparison of kidney imaging at contrast and non-contrast CT. Red arrows point to kidney regions.

In this work, we presented a fully automatic kidney segmentation algorithm on non-contrast CT images. It lies on three major contributions: 1) kidney regions are constrained by exploiting the segmented liver and spleen as well as body symmetry; 2) a probabilistic shape prior is constructed to handle the issue of kidney touching other organs; 3) efficient belief propagation is employed with the shape prior to extract the kidneys. Both quantitative and qualitative experiments demonstrated that our approach can accurately segment kidneys on CTC images.

2. METHODOLOGY In this section, we detail our algorithm to segment kidneys on non-contrast CT images. It consists of three major steps, data preprocessing, right kidney segmentation, and left kidney segmentation. 2.1 Data Preprocessing Liver and spleen are automatically segmented using Linguraru’s approach2. A probabilistic atlas was first built on the training dataset followed by registration with the patient image to coarsely determine the liver and spleen regions. The coarse segmentation was then enhanced by a geodesic active contour. In the next step, the intensity profile was estimated on the current segmentation, and used to remove outliers. To alleviate over-segmentation when kidneys touch the liver or spleen, we reduced the decision thresholds at the step of outlier removal. Finally, location corrections from the normalized probabilistic atlas were performed. 2.2 Right Kidney Segmentation 2.2.1 Kidney range determination After spleen and liver are segmented, we can use the segmented organs to roughly determine the kidney ranges. The right kidney is first segmented because it is adjacent to the liver and the large volume of the liver facilitates the determination of kidney range. Two key slice indices are identified from the segmented liver images as illustrated in Fig. 2. One is the slice (l) with the maximum area of the segmented liver, and the other one (h) is the most inferior slice containing the liver. The right kidney in the corresponding slices is annotated by red arrows.

Proc. of SPIE Vol. 8670 867005-2

We next determine the sub-image of the right kidney based on these two key indices. The top slice of the right kidney sub-image is experimentally chosen to l-10, and the bottom slice is h+50. The spatial ranges of the right kidney at x-y image plane are decided according to the spatial ranges of the human body as well as the ribs. The right kidney should only stay at the right side of the body, and the spatial region of the right body is determined based on the symmetry of the rib. Here, the rib is extracted by using the region growing algorithm as the rib has one of the highest intensity values in the abdomen. Finally, the sub-image of the right kidney is extracted by using top and bottom indices as well as the segmented rib.

(a) slice l

(b) slice h

Fig. 2: Two image slices determined by the segmented liver for extracting possible right kidney ranges. Green regions are the segmented liver and blue regions the segmented spleen. The right kidney is marked by red arrows.

2.2.2 Probabilistic atlas construction We selected five pairs of reference CT images and their segmented right kidneys to register with the right kidney subimages. We also extract subimages with the same size as the right kidney sub-images from the reference CT images and segmented kidney images. The reference CT subimages and the current patient subimages were then matched by affine14 and non-rigid15 image registrations. Remember that we can remove the liver regions in the reference CT images during image registration because the liver segmentation is available. Our experimental results demonstrated that this removal significantly enhanced the registration accuracy. Next, the right kidney sub-image is thresholded to a binary volume by choosing the intensity range corresponding to [1, 176] HU. The affine and non-rigid image registrations are performed again on the binary sub-image and the segmented kidney of the referenced CT images. Again, the liver regions are excluded from the binary sub-image. A probabilistic atlas is finally constructed by averaging the five registered kidneys as illustrated in the right image of Fig. 3.

Fig. 3: Process of right kidney atlas construction, where five left images are the registration images to the current patient images and the right image is the probabilistic atlas constructed on the five registration images.

Proc. of SPIE Vol. 8670 867005-3

2.2.3 Efficient belief propagation A distance field is computed on the probabilistic atlas3. The distance range is then normalized to [0, 1] and used as a probability field. Let Dist(p) be the normalized distance field, where p is an object point. We formulate the kidney segmentation problem as a labeling process in an image graph, which resorts to minimizing an energy function using the Markov random field.

(

)

(

E ( f p ) = ∑ p∈R 2 min P( p ) − f p , a + ∑q∈N ( p ) min λ f q − f p , b 1

1

)

(1)

where 0 ≤ fp ≤ 1, and a and b are truncation values to prevent over-smoothing. λ is a parameter to balance two terms in Eq. (1). P(p)=Dist(p)•δ(p), and

1 1 < I(p) < 176 otherwise 0

δ ( p) = 

(2)

Efficient belief propagation4 formulates the labeling process as the message propagation over the image graph, and it is chosen to minimize Eq. (1) due to its high efficiency. It involves two strategies to reduce the computation time. First, a multi-resolution image graph is built on the kidney sub-image. The belief messages can be quickly propagated in the coarse level graph, so as to reduce the time of message propagation in the fine-level graph. Second, note that both terms in Eq. (1) are represented as L1 norm. The belief messages at iteration t can be derived as a nested minimization convolution.

(

)

m tp→q = min h( f q ), min f p (h( f p ) + d )

(3)

m tp→q is the belief message that point p transmits to q at time t, and d is a constant cost.

( (

)

h( f p ) = min f p min P( p ) − f p , a + ∑s mst −→1q 1

)

(4)

3

Thus, Eq. (3) can be minimized through a generalized distance transform . Right kidney is segmented by choosing object points with fp>0.05 to ensure all kidney regions are included. The right kidney is finally refined by eliminating boundary points if they are included by the segmented liver. 2.3 Left Kidney Segmentation After the right kidney is segmented, we project it onto the left side of the body by reflecting it in the medial plane perpendicular to the x-y image plane and determined by the segmented rib. The size of the projected kidney is enlarged to make sure it contains the entire left kidney. Next, we follow the same strategy to build the probabilistic atlas and perform efficient belief propagation to extract the left kidney. Finally, we use the segmented spleen to refine the left kidney segmentation.

3. EXPERIMENTAL RESULTS We validated the accuracy of the kidney segmentation on five non-contrast CTC images with manual segmented kidneys as the ground-truth. We also chose 27 CTC cases to evaluate the stability of the proposed algorithm. Retrospective analyses of these images were approved by our Office of Human Subjects Research. 3.1 Accuracy Evaluation Six metrics established on liver segmentation2 were employed to evaluate the accuracy of the kidney segmentation. They are volume overlap (VO), Dice coefficient (DC), relative absolute volume difference (RA), average symmetric absolute surface distance (AS), symmetric RMS surface distance (SR), and maximum symmetric absolute surface distance (MS). Table 1 gives the results of segmentation accuracy on left and right kidneys. Note that the segmentation accuracy of the right kidney is slightly better than that of the left kidney. This is because the exclusion of the segmented liver improves the accuracy of the shape prior construction. Moreover, the liver was more accurately segmented than the spleen by using Linguraru’s method2.

Proc. of SPIE Vol. 8670 867005-4

Table 1. Accuracy evaluation on five non-contrast CT images with manually segmented kidneys as the ground-truth.

Kidney

VO(%)

DC(%)

RA(%)

AS(mm)

SR(mm)

MS(mm)

Right

80±5.2

89±3.3

10±5.1

1.9±0.59

2.8±0.93

10±2.9

Left

79±4.9

88±3.2

8±3.6

2.1±0.67

3.4±1.3

14±4.1

3.2 Stability Analysis Experiments were also carried out on 27 CTC cases to evaluate the stability of the kidney segmentation. 23 cases were successfully segmented. The left three columns in Fig. 4 illustrate three successful examples. In Fig. 4(a, e), the left and right kidneys are accurately segmented despite the fact that the left kidney touches the spleen. In Fig. 4(b, f), kidneys were still accurately segmented although the spleen was not correctly extracted. This is because the body symmetry as well as the segmented right kidney limited the spatial ranges of the left kidney. Consequently, both kidneys were successfully segmented. Fig. 4(c, g) present a difficult case in which the body symmetry was violated as left and right kidneys are unevenly located at different heights. Thanks to the segmented spleen, our segmentation algorithm was insensitive to this variance and could still segment kidneys accurately. Fig. 4(d, h) illustrates a typical failure case caused by the incorrect spleen segmentation. The left kidney was over-segmented and includes parts of the touching spleen. This is because the spleen and the kidney together were incorrectly regarded as a single kidney during the construction of the probabilistic shape prior, which generated the kidney over-segmentation. However, the right kidney was correctly segmented due to the segmented liver.

(a)

(b)

(c)

(d)

(e) (f) (g) (h) Fig. 4: Kidney segmentation results on four CTC datasets, where (a-d) are the original CT images, and (e-h) the segmentation results. Each column represents one patient. The segmented liver is visualized in red, the spleen in green, and the kidneys in light brown. Kidneys were well segmented in the left three columns, while the segmentation of the left kidney failed in the last column due to the incorrect spleen segmentation.

4. SUMMARY In this paper, we presented a fully automatic algorithm able to accurately and automatically segment kidneys on noncontrast CTC images acquired for screening to detect colon cancer. The kidneys, just as the liver and spleen, are important in abdominal diagnosis and, in our application, the detection of extracolonic findings from screening data. The probabilistic shape prior, body symmetry, and efficient belief propagation were employed to accurately localize kidneys and remove neighboring tissues with similar appearance. The accuracy of our kidney segmentation algorithm on noncontrast CTC datasets was high, as indicated in the Table. We also validated the robustness on 27 CTC additional cases,

Proc. of SPIE Vol. 8670 867005-5

and 23 datasets were successfully segmented. Experimental results confirm that kidneys can be accurately segmented if the liver and spleen are reasonably extracted. The segmented kidneys are critical to detect some clinically important extracolonic findings in kidneys, such as stones and lesions. The ability of the technique to work on CT images obtained without intravenous contrast, which is typically not given for CTC, is an important advance. The method shows great potential to assist with the automated analysis of full abdominal imaging data.

ACKNOWLEDGEMENTS This work was supported by the Intramural Research Program of the National Institutes of Health, Clinical Center.

REFERENCES [1] Pickhardt, P.J., Hanson, M.E., Vanness, D.J., Lo, J.Y., Kim, D.H., Taylor, A.J., Winter, T.C., and Hinshaw, J.L., " Unsuspected Extracolonic Findings at Screening CT Colonography: Clinical and Economic Impact," Radiology, 249(1), 151-159 (2008). [2] Linguraru, M.G., Sandberg, J.K., Li, Z., Shah, F., and Summers, R.M., “Automated segmentation and quantification of liver and spleen from CT images using normalized probabilistic atlases and enhancement estimation,” Medical Physics, 37(2), 771-783 (2010). [3] Maurer, Jr. C.R., Qi, R., and Raghavan, V., “A linear time algorithm for computing exact Euclidean distance transform of binary images in arbitrary dimension,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 265-270 (2003). [4] Felzenszwalb, P.F. and Huttenlocher, D.P., “Efficient Belief Propagation for Early Vision,” International Journal of Computer Vision, 70(1), 41-54 (2006). [5] Ali, A.M., Farag, A.A., and El-baz, A.S., “Graph Cuts Framework for Kidney Segmentation with Prior Shape Constraints,” Proc. MICCAI, 384-392 (2007). [6] Yuksel, S.E., Elbaz, A.S., and Farag, A.A., “A Kidney Segmentation Framework for Dynamic Contrast Enhanced Magnetic Resonance Imaging,” Journal of Vibration and Control, 13(9-10), 1505-1516 (2007). [7] Lin, D., Lei, C., and Hung, S., “Computer-aided kidney segmentation on abdominal CT images,” IEEE Transactions on Information Technology in Biomedicine, 10(1), 59-65 (2006). [8] Campadelli, P., Casiraghi, E., Pratissoli, S., and Lombardi, G., “Automatic Abdominal Organ Segmentation from CT images,” Electronic Letters on Computer Vision and Image Analysis, 8(1), 1-14 (2009). [9] Khalifa, F., Elnakib, A., Beache, G.M., Gimelfarb, G., ElGhar, M.A., Quseph, R., Sokhadze, G., Manning, S., McClure, P., and El-Baz, A., “3D Kidney Segmentation from CT Images Using a Level Set Approach Guided by a Novel Stochastic Speed Function,” Proc. MICCAI 6893, 587-594 (2011). [10] Freiman, M., Kronman, A., Esses, S.J., Joskowicz, L., and Sosna, J., “Non-parametric iterative model constraint graph min-cut for automatic kidney segmentation,” Proc. MICCAI, 73-80 (2010). [11] Cuingnet, R., Prevost, R., Lesage, D., Cohen, L.D., Mory, B., and Ardon, R., “Automatic Detection and Segmentation of Kidneys in 3D CT Images Using Random Forests,” Proc. MICCAI, 66-74 (2012). [12] Linguraru, M.G., Pura, J.A., Pamulapati, V., and Summers, R.M., “Statistical 4D graphs for multi-organ abdominal segmentation from multiphase CT,” Medical Image Analysis, 16, 904-914, (2012). [13] Boykov, Y. and Funkalea, G., “Graph Cuts and Efficient N-D Image Segmentation,” International Journal of Computer Vision, 70(2), 109-131 (2006). [14] Studholme, C., Hill, D.L.G., and Hawkes, D.J., “An Overlap Invariant Entropy Measure of 3D Medical Image Alignment,” Pattern Recognition, 32(1), 71-86 (1999). [15] Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., and Hawkes, D.J., “Non-rigid registration using free-form deformations: Application to breast MR images,” IEEE Transactions on Medical Imaging, 18(8), 712-721 (1999).

Proc. of SPIE Vol. 8670 867005-6