A Uni-parametric family of two-point third order iterative methods free from second derivatives: Substitute for Chebyshev’s method

D.K.R BABAJEE AND

MZ DAUHOO

Technical Report In Numerical Analysis Department Of Mathematics

25 TH M AY 2007

Table of Contents List of Figures

ii

List of Tables

iii

Abstract

iv

0.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

0.2

Univariate Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

0.3

Multivariate Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

0.4

Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Conclusion

23

Bibliography

24

i

List of Figures 1

Convergence results in terms of successful starting points for Test Cubic using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2

Convergence results in terms of mean iteration number for Test Cubic using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3

Convergence results in terms of successful starting points for Complex Cubic using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4

Convergence results in terms of mean iteration number for Complex Cubic using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5

Convergence results in terms of successful starting points for Even ODE using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6

Convergence results in terms of mean iteration number for Even ODE using IFEF family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

ii

List of Tables 1

Convergence properties and informational efficiency of the special members of the IFEF families: Univariate Case . . . . . . . . . . . . . . . . . .

2

Convergence properties of the special members of the IFEF family: Multiple Roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

6

9

Convergence properties of the special members of the IFEF families: Multivariate Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4

Convergence Results for scalar non-linear equations . . . . . . . . . . . . . 17

5

Key Results for Multiple Root Example. . . . . . . . . . . . . . . . . . . . 18

6

Convergence Results for system of non-linear equations . . . . . . . . . . . 22

iii

Abstract In this report, we present a uniparametric family of two-point third order iterative methods based on the approximation of the second derivatives in the Chebyshev method by a finite difference between two first derivatives. For scalar equations, we prove that the family converges cubically at simple roots and linearly at multiple roots by using classical Taylor expansion. We study the local convergence of the multivariate version of the family using Ostrowski’s technique based on the point of attraction and prove its third order convergence. Finally, we show both analytically and numerically that the family behaves like the Chebyshev method for large values of the parameter involved.

iv

0.1 Introduction For more than 300 years, the second order Newton method has remained the most common iterative method for finding approximate solutions to the non-linear equations of the form f (x) = 0. Third order iterative methods like the Halley and Chebyshev methods [8], despite their cubic convergence, are considered less practical from a computational point of view because of the costly second derivatives. However, during the last six years, many third order two-point iterative methods free from second derivatives have been derived and studied [2, 3, 5, 6, 7, 9, 14]. They can be obtained from quadrature formulae [2, 3, 7, 9, 14] or approximation to second derivatives [2, 3, 5, 6]. These methods require the function or its first derivative evaluated at two different points, usually using Newton’s method as the first step. Numerical experiments also confirmed they can be competitive to Newton’s method [2]. Ezquerro and Hernandez [5, 6] have derived a family of such third order methods free second derivative by approximating the second derivative in the Halley method with a finite difference between first derivatives. This family is a tributary of Varmann’s family [13]. The latter has been studied and analyzed thoroughly by Babajee and Dauhoo [3]. In the latter work, we showed that for large values of the parameter β, the Varmann family gives similar results to Halley’s method both analytically and numerically. In the present work, we apply the finite difference approximation to the second derivatives in the Chebyshev method in order to obtain a new uniparametric family of third order methods. We show that this family is in fact the Inverse Free version of the Ezquerro and Hernandez family. The paper is organized as follows: The convergence behaviour of the family at simple and multiple roots for solving scalar equations is investigated using classical Taylor expansion. We then extend the family to multivariate case and prove that it converges locally and cubically using the Ostrowksi technique [10, 12] based on point of attraction instead of classical Taylor expansion. Finally, we test the family by various numerical experiments and confirm that the family behaves as Chebyshev’s method for large values of β.

1

0.2 Univariate Case

0.2 Univariate Case 0.2.1 Derivation Let us define u(xk ) =

f (xk ) u(xk )f (2) (xk ) and L(x ) = . k f 0 (xk ) f 0 (xk )

The Newton NM method of quadratic convergence for solving scalar non linear equations is given by yk = xk − u(xk ). The Halley HM method is given by ¶−1 1 u(xk ). = xk − 1 − L(xk ) 2 µ

xk+1

(1)

Halley’s method require the costly second derivative evaluation. To avoid its computation, Ezquerro and Hernandez [5, 6] have approximated the second derivative by the following finite difference formula: f (2) (xk ) =

f 0 (zk ) − f 0 (xk ) , zk − xk

(2)

where zk = (1 − θ)xk + θyk , θ ∈ (0, 1], is convex linear combination. With this approximation (2), we get L(xk ) =

f 0 (xk − θu(xk )) − f 0 (xk ) . θf 0 (xk )

(3)

Substituting eq.(3) into eq.(1), we obtain the Ezquerro and Hernandez family which is xk+1 = xk −

(2θ −

1)f 0 (x

2

2θf (xk ) 0 k ) + f (xk − θu(xk ))

(4)

0.2 Univariate Case Now, the Chebyshev CM method is given by xk+1

µ ¶ 1 = xk − u(xk ) 1 + L(xk ) 2

(5)

If we substitute eq.(3) into eq.(5), we obtain a new family of iterative methods free from second derivatives: xk+1

· ¸ 1 1 0 −1 0 = xk − u(xk ) 1 + − f (xk ) f (xk − θu(xk )) 2θ 2θ

(6)

We next study the link between the new family and that of Ezquerro and Hernandez family. Since L(xk ) is small, substituting the series approximation µ ¶−1 1 1 1 − L(xk ) = 1 + L(xk ) 2 2 into eq.(1), we obtain the Chebyshev method which requires only one inverse calculation and can thus be termed as the Inverse-Free Halley method. The Ezquerro and Hernandez family can also be written as: xk+1 = xk − (1 + A(xk , θ))−1 u(xk ),

(7)

where A(xk , θ) =

1 0 f (xk )−1 [f 0 (xk − θu(xk )) − f 0 (xk )] . 2θ

A Taylor expansion of A(xk , θ) about xk yields £ ¤ f (3) (xk ) θ + O (θu(xk ))2 A(xk , θ) = −L(xk ) + u(xk ) 0 4 f (xk ) which can be considered as small because L(xk ), u(xk ) are small and θ ∈ (0, 1]. Thus, by substituting the approximation (1 + A(xk , θ))−1 ' 1 − A(xk , θ), 3

0.2 Univariate Case into eq.(7), we get back our new family defined by eq.(6). This means that the new family is the Inverse-Free version of Ezquerro and Hernandez family. We also notice that as θ → 0 in 1 eq.(??), we recover the Chebyshev method. By letting θ = , we can write the Ezquerro 2β and Hernandez family defined by eq.(4) as: xk+1 = xk −

f (xk ) ³ (1 − β)f 0 (xk ) + βf 0 xk −

´,

1 u(xk ) 2β

(8)

1 with β ≥ . The Ezquerro and Hernandez family is similar to the Varmann family which 2 is defined by eq.(8) with β 6= 0. The tributaries of Ezquerro and Hernandez family are the Arithmetic Mean Third Order Newton (AM) method, Mid-point Third Order Newton (MP) method and Halley method corresponding to the cases β = 0.5, β = 1 and β → ∞, respectively. The Inverse-Free Ezquerro and Hernandez (IFEF) family can be written as:  xk+1 = xk − u(xk ) 1 + β − β

³ f 0 xk −

´

1 u(xk ) 2β

f 0 (x

k)

 , β ≥ 1. 2

(9)

The cases β = 0.5 and β = 1 of this family are termed as the Inverse Free Arithmetic Mean Third Order Newton (IFAM) method and Inverse Free Mid-point Third Order Newton (IFMP) method, respectively. Definitely, the Chebyshev method is the special case β → ∞. We next prove that the IFEF family defined by eq.(9) is locally third order convergent at simple roots.

0.2.2 Order of Convergence Theorem 0.1 Let f be a real function. Assume that f (x) has first, second and third derivatives in the interval (a, b). If f (x) has a simple root x∗ ∈ (a, b) and x0 is sufficiently close to x∗ , then the IFEF family of iterative methods defined by eq.(9) has third order convergence

4

0.2 Univariate Case and satisfy the error equation: ek+1 = C(x∗ , β)

and constants cj = Proof.

e3k

µ ¶ ¡ 4¢ 3 2 + O ek , where C(x∗ , β) = 2c2 − 1 − c3 4β

(10)

f (j) (x∗ ) for j = 2, 3.. j!f 0 (x∗ )

Since xk = x∗ + ek , we have  xk+1 = x∗ + ek − u(x∗ + ek ) 1 + β − β

³ f 0 x∗ + e k −

1 u(x∗ 2β

f 0 (x∗ + ek )

´ + ek ) 

(11)

Using Taylor Series about ek , we have 1 1 (2) f (x∗ )e2k + f (3) (x∗ )e3k + ... 2! 3! 0 2 = f (x∗ )ek [1 + c2 ek + c3 ek + ...]

f (x∗ + ek ) = f (x∗ ) + f 0 (x∗ )ek +

(12)

Similarly, f 0 (x∗ + ek ) = f 0 (x∗ )[1 + 2c2 ek + 3c3 e2k + ...].

(13)

Eqs.(12) and (13) give f (x∗ + ek ) f 0 (x∗ + ek ) = ek [1 + c2 ek + c3 e2k + ...][1 + 2c2 ek + 3c3 e2k + ...]−1

u(x∗ + ek ) =

= ek [1 − c2 ek + (2c22 − 2c3 )e2k + ...], so that

µ ¶ 1 1 1 x∗ + e k − u(x∗ + ek ) = x∗ + 1 − c2 e2k + .... ek + 2β 2β 2β

and finally,

µ f

0

¶ 1 u(x∗ + ek ) x∗ + e k − 2β

5

(14)

0.2 Univariate Case "

µ

1 = f 0 (x∗ ) 1 + 2 1 − 2β

Ã

¶ c2 ek +

# µ ¶2 ! c22 1 + 3c3 1 − e2k + ... . β 2β

(15)

Using eqs.(13) and (15), we have µ 0

−1 0

βf (x∗ + ek ) f

¶ µ ¶ 1 3 2 x∗ + ek − u(x∗ + ek ) = β − c2 ek + 3c2 − 3c3 + c3 e2k + ... 2β 4β (16)

Substituting eqs.(14) and (16) into eq.(11), we obtain after simplification, xk+1 = x∗ + ek −ek [1 − c2 ek +

(2c22

−

2c3 )e2k

or

¶ ¸ · µ 3 2 2 c3 ek + ... + ...] × 1 + c2 ek − 3c2 − 3c3 + 4β

µ xk+1 = x∗ +

µ

2c22

3 − 1− 4β

¶ ¶ ¡ ¢ c3 e3k + O e4k

(17)

Eq.(17) is in fact the error equation for the IFEF family in eq.(10). Hence the proof is completed.

The information efficiency [2] is defined as EF F = q 1/d , where q is the

order of the method and d is the information usage, that is, the number of new pieces of information required per iteration. Table 1: Convergence properties and informational efficiency of the special members of the IFEF families: Univariate Case β Members 0.5

IFAM

1

IFMP

∞

CM

C(x∗ , β) 1 2c22 + c3 2 1 2c22 − c3 4

q

d

EF F

3

3

31/3 = 1.44

3

3

31/3 = 1.44

2c22 − c3

3

3

31/3 = 1.44

Table 1 indicates that the efficiency of the IFEF family is higher than the Newton’s method which is EF F = 21/2 = 1.41. We also note that the asymptotic error constant for Chebyshev method has obtained by making β → ∞ in eq.(10). In the following section 6

0.2 Univariate Case we prove that the IFEF family is only linearly convergent at multiple roots.

0.2.3 Multiple Roots Definition 0.1 [11] x∗ is a root of multiplicity m of the equation f (x) = 0 if f (x) = (x − x∗ )m g(x), where g(x) is continuous in an neighborhood of x∗ and g(x∗ ) 6= 0. If m is odd the root is said to be odd; if m is even, the root is even. Theorem 0.2 Let f be a real function and m-times continuously differentiable in the interval (a, b). If f (x) has a multiple root x∗ ∈ (a, b) of order m > 1 and x0 is sufficiently close to x∗ , then the family of iterative methods defined by eq.(9) is at least linearly convergent and satisfy the following error equation: ¡ ¢ ek+1 = Ω(β, m) ek + O e2k , where Ω(β, m) = 1 − M , M =

(18)

¢ 1 ¡ 1 1 + β − βσ m−1 and σ = 1 − . m 2βm

g 0 (x∗ ) 1 and assume that β > . By developing g(xk ) at xk = x∗ using g(x∗ ) 2m Taylor series (see [11]), f (xk ) can be written as: Proof.

Let η =

£ ¤ f (xk ) = g(x∗ )(xk − x∗ )m 1 + η(xk − x∗ ) + O((xk − x∗ )2 ) and 0

m−1

f (xk ) = m g(x∗ )(xk − x∗ ) so that 1 u(xk ) = m Now,

µ

· ¸ m+1 2 1+ η(xk − x∗ ) + O((xk − x∗ ) ) , m

¶ ¶ µ m+1 2 η ek + O(ek ) . 1+ η− m ³ f 0 xk −

´

1 u(xk ) 2β

f 0 (x) 7

(19)

0.2 Univariate Case · =σ

m−1

µ 1+

¶ ¸ m−1η m+1 m+1 2 ησ + − η ek + O(ek ) . m m2 σ m

(20)

Using eqs.(19) and (20), we get  1 + β − β ·

η m+1 βσ m−1 = M ek + − η − m m2 m

³ f 0 xk −

´

1 u(xk ) 2β

f 0 (x) µ

 u(xk )

m+1 m−1η m+1 ησ + − η m m2 σ m

¶¸ e2k + ... , (21)

so that ek+1 = (1 − M ) ek + O(e2k ). If we know the multiplicity of the root, then we can modify the family of iterative methods so that it is quadratically convergent at multiple roots like Newton’s method. [1]. Corollary 0.1 Under the conditions of Theorem 0.2 the modified family of iterative methods given by

 xk+1 = xk −

u(xk )  1+β−β M

³ f xk − 0

´

1 u(xk ) 2β

f 0 (xk )

,

has at least quadratic convergence and satisfy the following error equation: ek+1

· µ ¶¸ 1 η m+1 βσ m−1 m + 1 m−1η m+1 = − η− ησ + − η e2k + .. . (22) M m m2 m m m2 σ m

Proof.

Dividing eq.(21) on both sides by M we get 

f 1  1+β−β M

0

³ xk −

´

1 u(xk ) 2β

f 0 (x)

 u(xk )

¶¸ · µ 1 η m+1 βσ m−1 m + 1 m−1η m+1 = ek + − η− ησ + − η e2k + ... 2 2 M m m m m m σ m from which we obtain eq.(22).

8

0.3 Multivariate Case

Table 2: Convergence properties of the special members of the IFEF family: Multiple Roots β

Members

q

0.5

IFAM

1

1

IFMP

1

∞

CM

1

Ω(β, m) µ ¶m−1 ! 3 1 1 1 m− + 1− m 2 2 m à ¶m−1 ! µ 1 1 m−2+ 1− m 2m 1 (2m2 − 3m + 1) 2m2 Ã

Table 2 shows that the linear convergence of the IFVF family with their respective asymptotic error constant Ω(β, m). Having analyzed the IFEF family of iterative methods for solving scalar non-linear equations, we next consider its multidimensional version for the solution of systems of non-linear equations.

0.3 Multivariate Case Our motivation to extend these methods to multivariate case is due to Ortega and Rheinbolt [10] which say that methods which require second and higher order derivatives, are rather cumbersome from a computational viewpoint. Note that, while computation of F~ 0 involves only n2 partial derivatives ∂a F~i , computations of F~ (2) requires only n3 partial derivatives ∂a ∂b F~i which is in general an exorbitant amount of work indeed. In short the Jacobian F~ 0 is an n × n matrix with n2 values while the Hessian F~ (2) is an n × n × n matrix with n3 values. The Newton’s method in multivariate case is given by ~ (~xk ) = ~xk − F~ 0 (~xk )−1 F~ (~xk ). ~yk = N Let

µ ~zk =

1 1− 2β

¶ ~xk + 9

1 ~ N (~xk ). 2β

(23)

0.3 Multivariate Case The multivariate version of the IFEF family is given by ~ xk ), ~xk+1 = G(~ h i ~ (x~k ) − x~k ) , β ≥ 1 . = ~xk − F~ 0 (x~k )−1 F~ (x~k ) + β(F~ 0 (~zk ) − F~ 0 (x~k ))(N 2

(24)

The members of IFEF’s family require the computation of only one inverse and it is therefore cheaper than the Ezquerro and Hernandez family of iterative methods given by £ ¤−1 1 ~xk+1 = ~xk − (1 − β)F~ 0 (~xk ) + β F~ 0 (~zk ) F~ (~xk ), β ≥ , 2 which requires the inversion of two matrices. Instead of classical Taylor expansion, we are going to study the local convergence of the IFEF family using Ostrowski’s technique [10, 12] based on point of attraction. This technique gives more precision on the estimation of the rate of convergence. We shall use the following notations as in [10]: Rq (F, ~x∗ ), Qq (F, ~x∗ ) and OR (F, ~x∗ ), OQ (F, ~x∗ ) are the rate and quotient convergence factors and orders of a general iterative process F at ~x∗ . For their definitions, see [10]. However, we state the following important definition: ~ :D ~ ⊂
k = 0, 1, ...

(25)

~ of ~x∗ such that S ~⊂D ~ and, for any ~x0 ∈ S, ~ the iterates if there is an open neighbourhood S ~ and converge to ~x∗ . {~xk } defined by eq.(25) all lie in D We next prove the following lemma. ~ ⊂
10

0.3 Multivariate Case 1. If F~ (p−1) is Lipschitz continuous, that is,

then,

kF~ (p−1) (~v ) − F~ (p−1) (~u)k ≤ γp−1 k~v − ~uk,

(26)

° ° p−1 ° ° γ X 1 °~ ° p−1 F~ (j) (~u)(~v − ~u)j ° ≤ k~v − ~ukp . °F (~v ) − F~ (~u) − ° ° j! p! j=1

(27)

2. If F~ (p) is bounded, that is,

then,

Proof.

kF~ (p) (~u)k ≤ Kp ,

(28)

° ° p−1 ° K ° X 1 ° °~ p F~ (j) (~u)(~v − ~u)j ° ≤ k~v − ~ukp . °F (~v ) − F~ (~u) − ° ° j! p! j=1

(29)

The proof follows from [10, pp.81]. Briefly, set ~ = F~ (~v ) − F~ (~u) − − W

p−1 X 1 ~ (j) F (~u)(~v − ~u)j . j! j=1

By Taylor formula, we have Z

1

~ = W 0

i (1 − t)p−2 h ~ (p−1) F (~v + t(~v − ~u)) − F~ (p−1) (~u) dt(~v − ~u)p−1 (p − 2)!

By Schwartz inequality, Z

° (1 − t)p−2 ° ° ° (p−1) (p−1) ~ (~v + t(~v − ~u)) − F (~u)° dtk~v − ~ukp−1 °F (p − 2)! µ0Z 1 ¶ t(1 − t)p−2 ≤ dt γp k~v − ~uk k~v − ~ukp−1 , using eq.(26), (p − 2)! 0 γp ≤ k~v − ~ukp . p!

~k ≤ kW

1

11

0.3 Multivariate Case By Taylor formula, we can also write Z

1

~ = W 0

(1 − t)p−1 ~ (p) F (~u + t(~v − ~u))dt(~v − ~u)p (p − 1)!

By Schwartz inequality, µZ

1

~k ≤ kW 0

¶ (1 − t)p−1 dt sup kF~ (p) (~u + t(~v − ~u))k k~v − ~ukp (p − 1)! 0≤t≤1

Kp ≤ k~v − ~ukp , using eq.(28). p! We also consider two more theorems. ~ :D ~ ⊂ 1, then OR (F, ~x∗ ) ≥ OQ (F, ~x∗ ) ≥ q. If, in addition, the estimate ~ x) − G(~ ~ x∗ )k ≥ µ2 k~x − ~x∗ kq , for all ~x ∈ S, ~ kG(~ holds for some µ2 > 0, then OR (F, ~x∗ ) = OQ (F, ~x∗ ) = q. ~ ⊂
12

0.3 Multivariate Case We are now ready to prove the local and third order convergence of the IFEF family in the following theorem. ~ ⊂
(30)

~ (~x) defined by eq.(23) is well defined From Theorem 0.4, the Newton function N

and satisfies an estimate of the form ~ (~x) − ~x∗ k ≤ λk~x − ~x∗ k2 , for all ~x ∈ S ~1 , kN

(31)

~ x∗ , δ1 ) ⊂ S. ~ Consequently the mapping G(~ ~ x) defined by eq.(24) is ~1 = S(~ on some ball S ~2 = S(~ ~ x∗ , δ2 ) ⊂ S ~1 , where δ 2 ≤ δ1 /λ. also well defined on S 2 ~2 , using the Perturbation lemma [10, pp.45], we can show that F~ 0 (~x)−1 exists For any ~x ∈ S and is given by ~2 . kF~ 0 (~x)−1 k ≤ 2α, in S

(32)

~ x) − ~x∗ k ≤ kF~ (~x)−1 k kW ~ 1 k, kG(~

(33)

Then,

where ¶ ¸ · µ 1 ~ 0 0 0 ~ ~ (~x) − ~x). (34) ~ ~ ~ ~ (N (~x) − ~x) − F (~x) (N W1 = F (~x)(~x − ~x∗ ) − F (~x) − β F ~x + 2β By Mean Value Theorem for integrals and using ~ (~x) − ~x)2 = (N ~ (~x) − ~x∗ )2 + 2(N ~ (~x) − ~x∗ )(~x∗ − ~x) + (~x∗ − ~x)2 , (N 13

0.3 Multivariate Case eq.(34) becomes ~1 =W ~ 2 − 1W ~ 3 (~x∗ − ~x)2 − W ~ 4 (~x∗ − ~x)(N ~ (~x) − ~x∗ ) − 1 W ~ 4 (N ~ (~x) − ~x∗ )2 , W 2 2 where ~ 2 = F~ (~x∗ ) − F~ (~x) − F~ 0 (~x)(~x∗ − ~x) − 1 F~ (2) (~x)(~x∗ − ~x)2 , W 2 · µ ¶ ¸ Z 1 t (2) (2) ~3 = ~ (~x) − ~x) − F~ (~x) dt W F~ ~x + (N 2β 0 and

Z

1

~4 = W 0

µ ¶ t ~ (2) ~ (N (~x) − ~x) dt. F ~x + 2β

By Lemma 0.1 with p = 3 in eq.(27), we get kW2 k ≤

γ2 k~x∗ − ~xk3 . 6

(35)

Further by eq.(26) with p = 3, we obtain ° ° µ ¶ ° (2) ° t (2) ~ 3k ≤ ~ (~x) − ~x) − F~ (~x)° dt °F~ kW ~x + (N ° ° 2β 0 Z ¢ γ2 ¡ 1 ~ (~x) − ~xk t dt kN ≤ 2β 0 γ2 ~ ≤ (kN (~x) − ~x∗ k + k~x∗ − ~xk). 4β Z

1

(36)

By eq.(28) with p = 2, we have ~ 4 k ≤ K2 . kW

(37)

By the Triangle and Schwartz inequality and using eqs.(35), (36) and (37), the bound on ~ 1 is then given by W µ ~ 1k ≤ kW

γ2 γ2 + 6 8|β|

¶

γ2 ~ kN (~x) − ~x∗ k.k~x∗ − ~xk2 8|β| ~ (~x) − ~x∗ k2 , ~ (~x) − ~x∗ k.k~x∗ − ~xk + K2 kN +K2 kN 2 k~x∗ − ~xk3 +

14

0.3 Multivariate Case or using eq.(31), the above inequality reduces to µ ~ 1k ≤ kW

γ2 γ2 λγ2 δ2 λ2 K2 δ2 + + + λK1 + 6 8β 8β 2

¶ k~x∗ − ~xk3 .

(38)

Substituting inequalities (32) and (38) into inequality (33), we finally obtain ~ x) − ~x∗ k ≤ Ψ(α, λ, γ2 , K2 , δ2 , β)k~x∗ − ~xk3 , kG(~

(39)

where Ψ(α, λ, γ2 , K2 , δ2 , β) =

αγ2 αγ2 αλγ2 δ2 + + + 2αλK2 + αλ2 K2 δ2 3 4β 4β

Eq.(30) follows Theorem 0.3 which completes the proof. The estimate (39) is important since it indicates how the convergence proceeds close to ~x∗ . The larger Ψ, the worse the convergence is [10, pp.317]. Table 3 gives the local and third convergence of the special members of the IFEF family. Table 3: Convergence properties of the special members of the IFEF families: Multivariate Case β

Members

q

0.5

IFAM

3

1

IFMP

3

∞

CM

3

Ψ(α, λ, γ2 , K2 , δ2 , β) αγ2 αγ2 αλγ2 δ2 + + + 2αλK1 + αλ2 K2 δ2 3 2 2 αγ2 αγ2 αλγ2 δ2 + + + 2αλK1 + αλ2 K2 δ2 3 4 4 αγ2 + 2αλK1 + αλ2 K2 δ2 3

Having analyzed the family, we next compare it with other methods via some numerical experiments.

15

0.4 Numerical Experiments

0.4 Numerical Experiments 0.4.1 Scalar Non-Linear Equations Test Cubic We choose the simple cubic [4] f1 (x) = x3 + ln x, x∗ = 0.70470949025491 for which the logarithm restricts the function to be positive and its convex properties of the function are favorable for global convergence [4]. We test for 200 iterative methods of the family, equally spaced with ∆β = 0.5 in the interval [0.5, 100]. The starting points are equally spaced with ∆x = 0.01 in the interval (0, 10]. We denote the quantity ωc as the number of iterations from a starting point until convergence that is residual = |xk+1 − xk | smaller than 10−13 . A starting point is considered as divergent if it does not satisfy the above condition in at most 100 iterations and x ≤ 0 at any iterates. Further, we let N be Ns X ωi over a the number of staring points. We shall use the empirical mean value ω¯c = N1s i=1

sample of successful test values denoted by Ns for numerical comparison. 1001

6.9

1000.8 6.8

6.7

1000.4

Global convergence for all values of β tested

1000.2

Mean Iteration number

Number of Converging Points

1000.6

1000 999.8 999.6

6.6

6.5

Mean Iteration number converges to 6.23

6.4

6.3

999.4 6.2

999.2 999 0

10

20

30

40

50

β

60

70

80

90

6.1 0

100

10

20

30

40

50

β

60

70

80

90

100

Figure 1: Convergence results in

Figure 2: Convergence results in

terms of successful starting points

terms of mean iteration number for

for Test Cubic using IFEF family

Test Cubic using IFEF family

16

0.4 Numerical Experiments Figs.(1) and (2) shows the variation of the number of successful starting points and mean iteration number, respectively, with respect to β. We find that the family is globally convergent for all β tested. This is expected because the Chebyshev method is globally convergent for this problem. This is confirmed in Fig.(2) where we notice that as β grows large the mean iteration number ω¯c converges to 8.23 which is same as the Chebyshev mean iteration number in Table 4. More examples We now test the family with the following non-linear functions: f2 (x) = x2 − 2, x∗ = 1.41421356237310, x0 ∈ (−5, 5]. 2

f3 (x) = xex − sin2 (x) + 3 cos(x) + 5, x∗ = −1.20764782713092, x0 ∈ (−5, 5]. f4 (x) = (x + 2)ex − 1, x∗ = −0.44285440100239, x0 ∈ (−5, 5]. The stopping was same as before but x can be any value. We shall consider only the cases β = 0.5, β = 1 and β = 106 of the family and compare it with classical methods. Table 4: Convergence Results for scalar non-linear equations f1

Methods Ns

f2 ω¯c

Ns

f3 ω¯c

f4

Ns

ω¯c

Ns

ω¯c

β = 106

1000 6.23 1000 5.195

944

12.33

980

5.31

CM

1000 6.23 1000 5.192

977

12.13

995

5.26

IFAM

1000 6.71 1000

5.19

971

10.97

1000

5.14

IFMP

1000 6.47 1000

5.19

975

11.22

1000

5.09

NM

1000 8.82 1000

6.43

869

19.10

984

9.99

AM

1000 6.39 1000

4.64

877

10.59

947

6.31

MP

1000 5.92 1000

4.64

908

10.83

959

5.92

HM

1000 6.02 1000

4.64

969

15.37

999

6.87

17

0.4 Numerical Experiments Table 4 gives the convergence results of the iterative methods considered. Firstly, we observe that the IFEF family for large β gives almost similar results as Chebyshev method in terms of number of converging starting points and mean iteration number for test functions f1 and f2 . There are slight differences in the results for f3 and f4 . This is because we have to take in account the local convergence of the methods and the limitations of matlab in terms of round off error . On average, we notice that the third order methods are more efficient than Newton’s method. For test functions f3 and f4 , the inverse free version of the AM and MP methods give much better results than its predecessors. Example for multiple root We consider an example for multiple root f5 (x) = (x − 1)4 (x − 2)3 (x − 3)2 (x − 4). We choose x0 = 1.1 as the starting point which converges to the multiple root x∗ = 1 of order m = 4. The stopping criterion will be |xk+1 − xk | < 10−7 and if f 0 (xk ) = 0. We will only test for the main β. Table 5: Key Results for Multiple Root Example. Iterative methods

convergence iterations

error at the last iterate

β = 106

linear

31

9.7 × 10−8

CM

linear

32

3.8 × 10−8

IFAM

linear

34

7.3 × 10−8

IFMP

linear

33

6.8 × 10−8

Modified β = 106

quadratic

4

7.8 × 10−9

Modified CM

quadratic

4

2.1 × 10−6

Modified IFAM

quadratic

4

2.8 × 10−8

Modified IFMP

quadratic

4

1.7 × 10−9

18

0.4 Numerical Experiments Table 5 shows that there is almost no difference between the special members of the IFEF family and the Chebyshev methods for the multiple root example.

0.4.2 Systems of Non-Linear Equations Complex Cubic (Systems of 2 non-linear equations) We next consider the simple complex cubic [4] given by z 3 = 1,

z∈C

which can be stated as  F~E1 (x1 , x2 ) = 

 F1 F2



=

 x31

−

3x1 x22

−1

3x21 x2 − x32 

with x1,2 ∈ <. Eq.(40) has three solutions, namely, 

 = ~0

  1

,

(40)

  − 12 √ 3 2

,

 − 21 √

3 2

.

0 − A uniform grid of 66 × 66 with stepsize ∆x = 0.06 cast over the region (−2, 2) × (−2, 2) and the number of iterations until convergence counted for each gridpoint. A starting point was considered convergent if the L2 residual k~xk+1 − ~xk k2 has dropped below 10−13 in at most 500 iterations. For this problem the Jacobian F~ 0 (~x) is a 2 × 2 matrix with 4 partial derivatives and the Hessian F~ 2 (~x) is a 2 × 2 × 2 matrix with 8 partial derivatives to be calculated per iteration. So, we can consider the Chebyshev method.

19

0.4 Numerical Experiments 4356

9.4

4355.9

9.2

Global convergence

9

4355.7

Mean Iteration number

Number of Converging Points

4355.8

4355.6 4355.5 4355.4 4355.3

8.6 8.4 8.2 Mean Iteration number converges to 7.69

8

4355.2

7.8

4355.1 4355 0

8.8

10

20

30

40

50

β

60

70

80

90

7.6 0

100

10

20

30

40

50

β

60

70

80

90

100

Figure 3: Convergence results in

Figure 4: Convergence results in

terms of successful starting points

terms of mean iteration number for

for Complex Cubic using IFEF fam-

Complex Cubic using IFEF family

ily Fig.(3) shows that the family is globally convergent for most of the values of β tested and Fig. (4) shows that as β grows large, the mean iteration number tends to 7.69. Even ODE (Systems of n non-linear equations) We next consider the discretization of the non-linear second-order ODE 1 −x(2) (y) = τ x4 (y), y ∈ [0, 1], τ = , 2 with boundary conditions {x(0) = −2.5, x(1) = 2.5} using central differences and stepsize h. This yields the non-linear problem F~E2 (~x) = 0 with Fi = xi−1 − 2xi + xi+1 + h2 τ x4i ,

1
and the corresponding boundary terms. The above problem is similar in structure to some problems in process control, operations research and electrical circuit design. For the computational experiments, we choose n = 100 and thus h = 0.01. The discretisation results in a tridiagonal matrix with the non-linear elements on the diagonal and constant superdiag20

0.4 Numerical Experiments onal and subdiagonal entries. Therefore the Jacobian is a sparse matrix with 300 values to be calculated per iteration and is not computationally costly. Each starting vector is defined as ~x0 = (xr , xr , ..., xr )T uniformly, with xr varying according to the starting point. The xr , were equidistantly spaced over the interval (−2.5, 2.5), with a distance ∆x = 0.016 separating them. A point was considered converged if the L2 residual has dropped below 10−13 in at most 20 iterations. We do not consider the Chebyshev method over here because of the massive computation of the Hessian, a third order matrix with 30000 values. This problem actually shows the importance of the family. 276 6.35

Highest number 274

6.3 6.25

Mean Iteration number

Number of Converging Points

272 270 268 Number of converging points tends to 271

266 264 262

6.2 6.15 Mean Iteration number converges to 6.21

6.1 6.05 6

260

5.95

258

5.9

256 0

5.85 0

10

20

30

40

50

β

60

70

80

90

100

10

20

30

40

50

β

60

70

80

90

100

Figure 5: Convergence results in

Figure 6: Convergence results in

terms of successful starting points

terms of mean iteration number for

for Even ODE using IFEF family

Even ODE using IFEF family

The same phenomenon is observed in Figs.(5) and (6) for the Even ODE problem. In fact the number of successful starting points tends to 271 and the mean iteration number converges to 6.21 as β increases. Table 6 gives a numerical comparison of the methods in terms of number of converging points and mean iteration number for the Complex Cubic and Even ODE. We find that the special members of the IFEF family give more satisfactory results as compared to those of the Ezquerro and Hernandez family for the Complex Cubic.

21

0.4 Numerical Experiments

Table 6: Convergence Results for system of non-linear equations Methods F~E1 F~E2 N

Ns

ω¯c

N

Ns

ω¯c

β = 106

4356

4356 7.68

CM

4356

4356 7.69

IFAM

4356

4355 9.27

312 256

5.88

IFMP

4356

4356 8.82

312 267

5.89

NM

4356

4356 8.93

312 270

8.00

AM

4356

3940 6.02

312 284

6.19

MP

4356

4174 5.97

312 279

6.13

HM

4356

4356

5.6

22

312 272 -

-

-

-

6.41 -

-

Conclusion

In this paper we have derived the Inverse Free version of the Ezquerro and Hernandez family of third order methods, free from second derivatives. We have proved that this new family is third order convergent at simple roots and linearly convergent at multiple roots for scalar non linear equations. We also extended the family in order to solve systems of equations and proved the local third order convergence using the concept of point of attraction. The theory of IFEF’s family for β → ∞ is exactly the same as Chebyshev’s method. This point is confirmed by our numerical experiments which show that for large values of β, the family is in fact Chebyshev’s method. This indicates that there is no need to use the Chebyshev method which requires the computation of second order derivatives. One simply use the family with a suitable large β. This work gives an efficient substitute for Chebyshev’s method.

23

Bibliography [1] S. Amat, S. Busquier, S. Plaza, Review of some iterative root-finding methods from a dynamical point of view, SCIENTIA, Series A: Mathematical Sciences. 10 (2004) 3–35 . [2] D. K. R. Babajee, M. Z. Dauhoo, An Analysis of the Properties of the Variants of Newton’s method with Third Order Convergence, Appl. Math. Comp. 183 (2006) 659–684. [3] D. K. R. Babajee, M. Z. Dauhoo, Analysis of a family of two-point iterative methods with third order convergence, In: Simos, T.E., Psihoyios, G., Tsitouras, Ch. (eds.) International Conference on Numerical Analysis and Applied Mathematics 2006, WILEY-VCH Verlag GmbH & Co. KGaA, Greece, 2006, pp.658–661. [4] M. Drexler, Newton Method as a Global Solver for Non-Linear Problems, Ph.D. Thesis, University of Oxford, 1997. [5] J.A. Ezquerro, M.A. Hernandez, A uniparametric Halley-type iteration with free second derivative, Int. J. Pure Appl. Math. 6 (2003) 103–114. [6] J.A. Ezquerro, M.A. Hernandez, On Halley-type iterations with free second derivative. J. Comp. Appl. Math. 170 (2004) 455–459. [7] M. Frontini, E. Sormani., Third-order methods from quadrature formulae for solving systems of nonlinear equations, Appl. Math. Comput. 149 (2004) 771-782. [8] J. M.Gutierrez and M. A. Hernandez, A family of Chebyshev-Halley type methods in Banach spaces, Bull. Austral. Math. Soc. 55 (1997) 113–130. [9] H. H. H. Homeier, Modified Newton method with cubic convergence: the multivariate case, J. Comput. Appl. Math. 169 (2004) 161–169. [10] J. M. Ortega and W. C. Reinbolt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970. [11] N.Osada, An optimal multiple root-finding method of order three, J.Comput. Appl. Math. 51 (1994) 131–133. 24

[12] A. Ostrowski, Les points d’attraction et de repulsions pour l’iteration dans l’espace a ` n dimensions, C.R. Acad. Sci. Paris 244 (1957) 288–289. [13] O. Varmann, High Order Iterative Methods For Decomposition-Coordination Problems. Okio Technologinis IR Ekonominis Vystymas Technological and Economical Development of Economics, 2006. 7, 56–61 (2006). [14] S. Weerakoon and T.G.I. Fernando, A Variant of Newton’s Method with Accelerated Third Order Convergence, Appl. Math. Lett. 13 (2000) 87-93.

25