An Adaptive Extension of Combined 2D and 1D-Directional Filter Banks Yuichi Tanaka

Masaaki Ikehara

Truong Q. Nguyen

IS Dept., Utsunomiya University Utsunomiya, Tochigi, 321–8585 Japan Email: [email protected]

EEE Dept., Keio University Yokohama, Kanagawa 223–8522 Japan Email: [email protected]

ECE Dept., University of California, San Diego La Jolla, CA 92093–0407 USA Email: [email protected]

Abstract— In this paper, we propose an extension of combined 2D and 1D-directional filter banks (TODFBs) with adaptive approach in their 1D-directional stages. TODFBs show better performance in image coding and denoising compared to the traditional wavelet transform (WT), however, they still have possibilities to improve their performance in image coding by adopting the adaptive directional WTs for their 1D-directional stages. An efficient method to determine the transform directions is also proposed. In image coding results, our proposed filter banks gain PSNR and visual quality improvements compared with the WTs and the non-adaptive TODFBs.

I. I NTRODUCTION The traditional scheme to realize multiresolution image representation (MIR) is to apply 1D filters separately in horizontal and vertical directions, commonly referred to as “separable” transform. In contrast, “nonseparable” transforms consist of 2D filters and 2D downsampling matrices which cannot be factorized into 1D filter/downsampling pairs. The traditional wavelet transform (WT) is categorized as a separable transform, which is used in various applications [1], [2]. However, it has poor diagonal orientation selectivity since frequencies with different orientations are gathered into one subband in each resolution. For example, in image coding for low bit rates, reconstructed images often have blurred regions for diagonal orientations. One method was proposed to not discard high-frequency regions for image coding at low bit rates. It needs only 1D WT filters and is based on transformation along the “curves” in images [3]–[5]. In other words, 1D filters are rotated (or skewed) to fit lines in images (such as edges between objects and background). We denote this type of WTs as “AD WT” (acronym of Adaptive Directional WT) hereafter. First, the AD WT transforms an original image with the vertical downsampling matrix Mv = diag(1, 2), and then transforms with the horizontal one Mh = diag(2, 1) (or vice versa). After both downsamplings of the AD WT, one has four subbands corresponding to the traditional WT’s LL, LH, HL, and HH. Obviously the LL subband can be transformed recursively, thus finally the quad-tree MIR is obtained. The AD WTs are suited for the conventional quadtree coders. By transforming along the curves in images, the AD WTs preserve both high-frequency and low-frequency information even in low bit rates where the separable WTs often yield blurred artifacts. However, AD WTs have some drawbacks compared to the separable WT. The main drawback is the computational cost to determine the curves in images. At each pixel, transforms are needed for several directions to decide the direction which maximizes the energy in LL subband (i.e. minimizes the energies in LH, HL and HH subbands). To overcome the previously mentioned problems, the authors proposed new combinations of 2D and 1D-directional (TOD) FBs based on 2D nonseparable FBs and the directional WT transforming fixed (non-adaptive) directions (denoted as FD WT hereafter) [6], [7].

978-1-4244-3828-0/09/$25.00 ©2009 IEEE

They have two patterns in the choice of 2D FBs, shown in Fig. 1, with the same purpose: The first stage in the TODFB extracts the curves in the subbands by the 2D FBs, and the second stage reduces the redundancy in these curves by the FD WTs. They preserve both lowand high-frequency information even after the quantization/truncation of many transformed coefficients. However, one needs more accurate curve transformation such as the AD WTs in image coding. Therefore, we extend the structure of TODFBs in this paper by utilizing the structure of the AD WTs. Furthermore, the determining process of transform directions is different from those of the AD WTs. The proposed FBs are efficient and provide accurate edge information even in low bit rate image coding. II. AD WT The AD WTs [3]–[5] are based on the combination of 1D downsampling and transforming by 1D WT filters along the 2D directions. Obviously lines in images, called “curves” in this paper, often flow in diagonal orientations. A curve has similar information to its component, hence, the 1D filtering along the curve is expected to concentrate the energy into the low-frequency subband. Some of AD WTs are based on the lifting implementation of the separable WTs [8]. We use this scheme for our proposed combination. The directional lifting steps are illustrated in Fig. 2. Let x(m, n) denote the pixel value at (m, n). A prediction step for a direction θ with the vertical downsampling [4] is represented as h(m, 2n + 1) = x(m, 2n + 1) − P (m, 2n)

(1)

where h(m, 2n + 1) represents a highpass branch of the directional lifting step and P (m, 2n) = pi (x(m − tan θ, 2(n − l)) + x(m + tan θ, 2(n + l + 1))

(2)

in which pi is a coefficient for this prediction step and l is a nonnegative integer. An updating step is given by l(m, 2n) = x(m, 2n) + U (m, 2n + 1)

(3)

where l(m, 2n) represents a lowpass branch and U (m, 2n + 1) = ui (h(m − tan θ, 2(n − l) − 1) + h(m + tan θ, 2(n − l) + 1)),

(4)

in which ui is an updating coefficient. Clearly these lifting steps are perfect reconstruction and can be cascaded with other lifting steps similar to the separable WTs. In this paper, we use the lifting coefficients of the 9/7 WT in [8]. We define the notations of the transform directions by the AD WT as the relative pixel position from the pixel to be transformed. Some typical directions are illustrated in Fig. 3 where the direction for the

2193

Fig. 1. Two TODFB frameworks where SepWT and DirWT referred to separable WT and directional WT, respectively. (Left) Q-TODFB (introduced in Section III-A). (Right) D-TODFB (introduced in Section III-B).

(1, 3) (1, 2) Interpolation required Downsampling Direction

(0, 1) (1, 1)

Row to be transformed

(-1, 1)

(-1, 2) Interpolation required

Fig. 2. Directional lifting steps for the directional WT. (Left) prediction step. (Right) updating step.

(-1, 3)

Fig. 3.

separable WT is defined as (0, 1). Note that the even (odd) row to be transformed requires neighbored odd (even) rows in each lifting step for perfect reconstruction. Therefore, the directions (1, 2), (-1, 2), etc. cannot be transformed without interpolating pixels. III. TODFB S In [6], [7], the authors introduced combinations of 2D FBs and FD WTs to yield efficient MIRs. The TODFBs perform well in applications of nonlinear approximation, image coding and image denoising. They are based on two stages: extracting “curves” in subbands by 2D FBs, and reducing energy in the curves by FD WTs. They have two patterns regarding the passband supports of 2D FBs, called Q-TODFB and D-TODFB, which are introduced in this section. A. Q-TODFB uses 2D FBs with quincunx downsampling factor Q =  Q-TODFB  1 1 (thus it is called Q-TODFB) whose passbands are two 1 −1 well-known variations: “diamond” and “fan” FBs. Indeed the fan (diamond) FB can be obtained by modulating the diamond (fan) FB by e−jπ . Therefore, these two FBs are fundamentally the same. The Q-TODFB decomposes an input subband by the quincunx FBs, and then performs FD WTs. Its framework is shown in Fig. 4. Obviously the subband 0 corresponds to the LL subband of WTs, hence it can be decomposed recursively by the Q-TODFBs or the separable WTs.

Directions for the directional WT.

The Q-TODFBs share the partial framework with the nuqDFB in [9]. The main difference of these two FBs is compatibility of their quad-tree MIR to that of the WT. The Q-TODFB can yield a WT-compatible MIR, whereas the nuqDFB have its typical MIR. As a result, the Q-TODFB will be embedded into an already existing framework using the WT such as compression and thus, they can be considered as a good alternative of the WT. B. D-TODFB D-TODFB is based on the finer WT proposed in [10], which decomposes high-frequency subbands yielded from the 1-level WT decomposition by diagonally quadrant FBs. In the D-TODFB system, the decomposed subbands are further transformed by the FD WTs. The framework is illustrated in Fig. 5. Similar to the Q-TODFB, the D-TODFB can perform recursively for the low-frequency subband. Furthermore, it is completely a post-processing of the traditional WTs. IV. A DAPTIVE E XTENSION OF TODFB S In this section, we extend the previously proposed TODFBs to those having the AD WT stage. Furthermore, an efficient method to determine the transform directions in the AD WT stage is also shown. It avoids a problem on the AD WTs about the computational costs by utilizing the property of the TODFBs.

2194

(-1, 2): Main direction

Q Q

LL Subband

(-1, 1)

(-1, 4)

Subband 0

Q Trans. direction: (0,1)/(1,0)

Subband 1

Q Q

Trans. direction: (-1,1)/(1,1)

Subband 2

Q Subband 3

Trans. direction: (1,1)/(-1,1)

Fig. 4.

Curve range

Directional WT Stage

Quincunx FB Stage

Fig. 6.

Curve range of the HH subband in the D-TODFB.

Q-TODFB decomposition.

with interpolation and the one with the alternative integer pixels is very similar.

LL Subband

H

Subband 1

B. Reduction of The Curve Calculation Cost

V

LH H

V

Subband 2

Subband 3

V

H

HL V

H

Subband 4

Subband 5

H

V

HH H

Diagonally Quadrant FB Stage

Subband 6

V

Directional WT Stage

Fig. 5. D-TODFB decomposition. (↓ H), (↓ V) are the horizontal and vertical downsamplings, respectively.

A. AD WT Stage in TODFBs The FD WT stages in TODFBs can be replaced by the AD WT ones for the adaptive TODFBs. With this replacement, they can transform along more accurate directions of curves. Unlike the conventional AD WTs, the adaptive TODFBs can restrict a set of direction candidates since subbands after transforming by 2D FBs already have direction-limited curves. For example, we consider the subband 6 in the adaptive extension of D-TODFBs. Before the AD WT stage, the main curve direction becomes (−1, 2) (see Fig. 6). In the non-adaptive D-TODFBs, the FD WT transforms along the direction or an alternative direction such as (−1, 1) [6]. However, curves in these subbands still have some ranges from (−1, 1) to (−1, 4) for the center curve direction (−1, 2). Consequently, the AD WT stage in the adaptive TODFBs transforms adaptively along the curves in these restricted ranges. Furthermore, we avoid to use interpolation filters to yield subpixel accuracy of curves. Instead, alternative curves using only integer pixels [5] are considered since it is difficult to design the optimal interpolation filter and the performance between the directional WT

For the AD WTs, the curve directions could vary in a wide range. In the previous subsection, we prove some directions can be reduced in the adaptive TODFBs. Additionally, further reduction of the calculation costs is introduced in this subsection. There is a well-known fact that most images have their essential information in low-frequency subbands, whereas high-frequency ones correspond to detail information. In other words, the high-frequency subbands have many “black” pixels and few pixels representating curves. In the viewpoint of the reduction of energy in these subbands, adaptive transforming for black pixels are quite redundant and it hardly affects the final coding performance. Therefore, we reduce the calculation costs by using sum of absolute values (SAV) of a block which contains one curve. The size of the blocks can be determined arbitrarily. The curve determining process is formalized as follows: Step 1Calculate SAVb , defined as the SAV of a block b ∈ S in the target subband S. Step 2If SAVb < T h where T h is an arbtrary threshold, the curve determining process is stopped; the block b is transformed by the separable vertical (or horizontal) WT. Step 3If SAVb ≥ T h, there exists a curve in the block. Step 4Determine the curve direction Db in b. Db is set as k if k satisfies arg min(|Elk | + |Ehk |), where Elk and Ehk are the energies of the low-frequency subband and the high-frequency one after transforming along the direction candidate k, respectively. Finally, a transform direction map is yielded for a target subband. If “0” is used to represent a transform direction for the separable WT, a map contains a lot of “0” values. Therefore, a bit budget of the map can be significantly reduced by using any entropy-coding such as run-length or variable-length method. V. E XPERIMENTAL R ESULTS In this section, the proposed adaptive TODFBs are used for image coding. We used SPIHT image transmission algorithm to encode multiresolution images [11]. We compare the proposed FBs with the traditional separable WT and the non-adaptive TODFBs. All transforms used 5-level decomposition. For both Q-TODFBs, we set a 2-level decomposition for the quincunx FB/directional WT pair and a 3-level one for the separable WT (denoted as Q-TODFB(2, 3)

2195

TABLE I I MAGE C ODING R ESULTS (PSNR (dB))

bpp 9/7 WT Q-TODFB D-TODFB Adaptive Q-TODFB Adaptive D-TODFB bpp 9/7 WT Q-TODFB D-TODFB Adaptive Q-TODFB Adaptive D-TODFB

Barbara 0.1 0.2 23.72 26.02 24.17 26.61 24.33 26.62 24.26 26.75 24.53 26.86 Zoneplate 0.1 0.2 9.91 10.42 10.18 11.31 10.17 12.03 10.21 11.45 10.23 11.95

0.3 27.66 28.08 28.07 28.24 28.34

0.4 29.44 29.50 29.51 29.67 29.91

0.5 30.64 30.73 30.65 30.94 31.12

0.3 11.10 13.29 13.45 13.75 13.75

0.4 11.83 14.50 14.56 15.13 14.99

0.5 12.94 15.79 15.82 16.56 16.38

in [6]). For both D-TODFBs, their two highest frequency subbands are decomposed by the diagonally quadrant FB/directional WT pair (denoted as D-TODFB(2, 3) in [6]). Test images Barbara and Zoneplate are used in this paper since they have a lot of rich textures. The PSNR comparison in various bitrates is shown in Table I. Clearly both of adaptive and non-adaptive TODFBs outperform the separable WTs. Furthermore, in middle bitrates, the adaptive TODFBs still keep high PSNRs. The cropped reconstructed images are shown in Fig. 7. The separable WT discards many textured regions. The adaptive and non-adaptive TODFBs preserve rich textures in the original image, however, for the sharp lines, the adaptive TODFBs (especially the adaptive Q-TODFB) have more accurate information than that of the non-adaptive ones. The image coding algorithm using the nuqDFB has been proposed in [12] and it outperforms the SPIHT with the 9/7 WT. However, the algorithm in [12] requires adaptive morphological operations which will require more computational costs than the traditional SPIHT. Furthermore, as previously mentioned, the nuqDFB does not have a compatible MIR to the quad-tree MIR of the WT. The existing application resources using the WT should be modified for using the nuqDFB. In contrast, the MIRs yielded from the TODFB are able to use these resources as-is. VI. C ONCLUSION In this paper, we propose the adaptive extension of the TODFBs. The proposed FBs show objective measure and visual quality improvements compared to the separable WT and the non-adaptive TODFBs. Furthermore, the calculation costs to determine the transforming direction are lower than the AD WTs. ACKNOWLEDGEMENT This work was supported by JSPS (Japan Society for the Promotion of Science), and also Global COE Program “High-Level Global Cooperation for Leading-Edge Platform on Access Spaces (C12)” at Keio University. R EFERENCES [1] P. P. Vaidyanathan, Multirate Systems and Filter Banks. NJ: PrenticeHall, 1993. [2] G. Strang and T. Q. Nguyen, Wavelets and Filter Banks. MA: WellesleyCambridge, 1996. [3] D. Wang, L. Zhang, A. Vincent, and F. Speranza, “Curved wavelet transform for image coding,” IEEE Trans. Image Process., vol. 15, no. 8, pp. 2413–2421, 2006.

Fig. 7. Reconstructed images on 0.25 bpp (Barbara). From left to right, top to bottom: Original, 9/7 WT, Q-TODFB, D-TODFB, Adaptive Q-TODFB, and Adaptive D-TODFB.

[4] W. Ding, F. Wu, X. Wu, S. Li, and H. Li, “Adaptive directional liftingbased wavelet transform for image coding,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 416–427, 2007. [5] C.-L. Chang and B. Girod, “Direction-adaptive discrete wavelet transform for image compression,” IEEE Trans. Image Process., vol. 16, no. 5, pp. 1289–1302, 2007. [6] Y. Tanaka, M. Ikehara, and T. Q. Nguyen, “Multiresolution image representation using combined 2-D and 1-D directional filter banks,” IEEE Trans. Image Process., vol. 18, no. 2, pp. 269–280, 2009. [7] ——, “A new combination of 1D and 2D filter banks for effective multiresolution image representation,” in Proc. ICIP’08, 2008, pp. 2820– 2823. [8] I. Daubechies and W. Sweldens, “Factoring wavelet transforms into lifting steps,” Journal of Fourier Analysis and Applications, vol. 4, no. 3, pp. 247–269, 1998. [9] T. T. Nguyen and S. Oraintara, “A class of multiresolution directional filter bank,” IEEE Trans. Signal Process., vol. 55, no. 3, pp. 949–961, 2007. [10] Y. Lu and M. N. Do, “The finer directional wavelet transform,” in Proc. ICASSP’05, 2005, pp. IV–573–IV–576. [11] A. Said and W. A. Pearlman, “A new, fast, and efficient image codec based on set partitioning in hierarchical trees,” IEEE Trans. Circuits Syst. Video Technol., vol. 6, no. 3, pp. 243–250, 1996. [12] Y. Liu, T. T. Nguyen, and S. Oraintara, “Embedded image coding using quincunx directional filter bank,” in Proc. ISCAS’06, 2006, pp. 4943– 4946.

2196