Jorge Nocedal.

# Projected Hessian updating algorithms for nonlinearly constrained optimization online

. (page 3 of 4)
Online LibraryJorge NocedalProjected Hessian updating algorithms for nonlinearly constrained optimization → online text (page 3 of 4)
Font size -44-

Theorem 4.3: Suppose "pdate Rule 1 Is user! in Aisorithm 4.1.
Choose a value v "> in condition (4.3). Then

\/re(0,n,3e>0, 5>0,ri>0, such that if He II _< e ,

II B -H^ll ,^ < 5, and n is used to define condition (4.3), then

n e^+^W < nie^.^ll , k ? 1 , (4.21)

i.e. , xj^ > x^ at least at a two-step 0-linear rate.

Proof: We prove the result for the PSB and OFP cases, leaving the

BFGS case as an exercise. Let a,a,,a2 be the constants from Lemma ^.3,

_ o

and define y^, M as in Corollary 4.2. Let a^ = II Mil ^a^. Let 3 be such

that

XII < B 11X11^

for any n X n mtrix X. Let ic = 2liH;'^ll , 5 = 1/ (211 H^^^!!) , so that, bv the
Banach lemma,

X-H,^ll < 5 => II X ^11 < K . (4.22)

Let E,,C, be as in Theorem 4.1, using this definition of 0, 5 > 0, n > to be
chosen small enough that

-45-

e < e ^/C^

e < ÂŁ9/0^^

6 < 5/(23)

C^(e + 2B6 ) < r

Mil - {209(1 + n)C, e + niiG^ii } < 1

'1

â€˘â€˘ â–  4C9a + n)C,e ,

(2a, 6 +09) ( : ^ + n(i +-] IIG^Il) < 5 .

(4.23)

Such a choice of e ,6 ,n is clearly possible. Now let 11 e^!l < e,
II B -H^ll X, < 5. We will establish by induction that

(i) IIB-H*II^ < 26 , .1 > .

(ii) II eyi < C^lle^.^ll , 1^1.

(iii) lle^^ll < rllej_-Jl , j > 1 â€˘ . (4.24)

(i V ) If_ the update formula is used at iteration j ,

then IIB^^^-H^IIm < [(1 - aO ^)^^- + a ^y jl" B .-H^ll ^ + a9Y ^

else B :j+i = B . , i > .

Note that (iii) is the desired result.

For j = 0, (i) is true by assumption and (ii), (iii) are vacuous.
For (iv), we need only verify the inequality assuming the update
formila is used to obtain B^. Because II e^H < E < t -^ and H B^ II < < ,
Theorem 4.1 applies (with k = 1), giving

-46-

e^ll

Note that (1 -ct6^ )'â– '-< 1 - a ^. Now from (4.24) (iv), we have

\.+r"*"M* [(1 -ote2_)i/2 +ci^Y,,.]llB^_-H^il +ajy^^_ ,

where, proceeding as before to apply Corollary 4.2, and usin as i + " .

S, II

'^i

Thus, using either (4. la) or (4.1b) for the definition of Sj^, and with
U|^ defined by (4.12), we obtain

(jj^ â€˘Â»â–  as i + Â» .

Now from Theorem 4.1(iii), for i large enough,

â– 52-

X 1 1 1 u

and hence bv ( 4. 2 4) (i 1) , (ill') ,

Thus

^k = li r"^ n as i > Â» ,

i "^k.-l" ,

I.e. , {Ci wJ NJ â€”

> >

tsj â€”

c

wi wi

ui ui \yi Ln

Ui Ui Ui Ui

O* OS CT 0^

ON

i> X>

^ ^

-s

- -

C^ ^ O^ o^

ON o^ o^ a^

^ ^ ^ ^

â– ^

NJ 00

3 â€”

U) LJ UJ UJ
H- O^ NJ NJ

J> ÂŁ^ X> i>

VD â€” â€” UJ

Ul U1 Wl UI
N) U) W â€”

UI +
l'

o

â€” rsj

U b

P! PI

1 1

3

ISJ

3

3

C

N3 00

1 1

00 O vTi â€”

c=: p: m r=3
1 t 1 1

O â€” CT^ C

p] p: m p:
1 1 1 1

00 O -o ÂŁ-

PI p] n m
1 1 1 1

CC

.- â€” O r^J

NJ < fSJ

o o o o

^

iâ€” ho

Ui LP

^ ^J -^ -^

-J -J ^ ^

i^ ui i-n ÂŁ>

ON

X> ÂŁ-

^ ^

^

CO 00 00 03
Ui k^ ui m

00 03 OC 00
Ui ui iÂ»n ui

On On O^ Ui
www

J> *^ i> UJ

J> UJ U) -^

(SJ 00

O O

L.J

II
rsj

0=

r-
p:

2

rO N3 PO LJ

n p: m n

yi ri :^ T=i

O NO NC Ul

P3 PI m p:

o

ON J>
P] PI

ISJ

VJ5 Â«

â€” ,- â€” â€”

â€” â€” â€” â€”

â€” â€” â€” ^

J> ^ ÂŁ-. ^-'

O

X> f>

^

w w w w

WW w^

Ui J>

Ui Ui ^ IJ1

i-n ui UI Ln

ON CT* ON ON

Ln vO

I> ÂŁ>

'^

. -^

."'^ â– ''^.

â€” vo

c> c^ cr o^

0^ O^ O O^

â€˘^ -^ -J -J

OS O â–

A-n "â€” â€ž

::.

-'

J> oJ

(^ L-i Lo ro

U> UJ LJ K)

i-n LD Ui Ln

ÂŁ> 00

o o

3
II
UÂ«

3
II

C

i

-a

wJ

â€” 'J

!> J> ^ Ln

Ui m Un 03

00 00 ^ â€”

^ 00

[SJ *Â»J

O -J

1 1

- vO

\0 ^O vO ^

n rr: pj n
1 1 1 1

â€” â€” ^ o

p; p: p: P!
1 1 1 1

-J X> O UJ

â€” PI p: PI
1 1 1 1

NJ N> SJ O

PI p]
1 1

nÂŁ> no

ÂŁ- SO

PI P5

: 1

N3 J>

â€”

z

^ ^

CO 00 00 OS

03 X 00 -^

ON OS ON OS

CO - J

U> Lf>

^O ^ÂŁ1

O '^ ^ sO

uo o ^ o

^ '^ -^ -O

NO 00

\0 i>

^

-3

2:

00 00

'^ *J ^J -vl

â– ^ - J -^ 00

Ui Ui ui Ln

kJ

tsJ L-J

w

\0 SJ 00 ^

P3 pj M rn

i' !T> O ISJ

PI PI P3 PI

(jJ W ^- tsj

PI P] P5 P3

ro o

m PI

vC â€”

VO H-

\0 ^ vO ^o

iÂŁ3 ^ sC â€”

â€˘^D â– â– ^ \D â– -0

so NO

55

w ^

Sj NJ NJ â€”

ro ro NJ H-

^- â€” tsj NJ

ISJ

Ui Ui

uj UJ uj n:

uj uj uj ro

N3 |NO W UJ

â€” +
V

sC

UJ ISJ
â€” OS

c o

ON VO
PI PI
1 1

D
1

3
II

(SJ

T3
C

2

i

c
o

*^ vO "^ vO

â€” â€” â€” o

â€” l^ Â»â€” Ul

o o <

vO 00 NJ Ln

ro ON

PI W
1 1

O â€”

O Ln UJ Ui

PI m w rr
1 1 t 1

o o o o

Ln cjv i* J>

p: PI PI PI
t 1 t 1
^ ^ ^ ^

1^ -^ *Â» Ln

P! PI m p:
1 1 1 1

^j2 \C ^ "â€”

\o o

^ V.O ^C O

O vÂŁ> ^ ^C

ro NJ ro u)

O Ui

Ul Ul

-3

p:

â€” Isj

o o o o

C C O O

1^ UJ UJ x>

â€” 0^

Â«; uj

II

O

00 \o

00 09 00 00

00 00 03 03

â€” " ^ I^

vO i^

O O

^

^

5i

fJN i-

â€” _ ^ ^

â€” â€” O â€”

â€” ON NJ Ui

â€” â€”

OS ÂŁ^

Ln CT^ Ui UJ
P5 PI PI PI

â€˘^ UJ ON -C*
PI PJ P5 PI

O \0 C3N x>

m p: p: p:

O NO

m p:

Ul UJ

p: p:

^O vO

>X) \0 â€” vC
w w o ^

vO vO â–şâ€” vÂŁ

w W O w

00 *0 \Ci \ÂŁ>

NO NO

CC

^ -o

00

3
II

-3

PI

EH

o

o o ^O o

o â€” o â€”

1^ Ui 1J1 i_n

â€” â€”

â€” Z 5 ^

â€” ISJ â€” ro

ON On O* ON

N> NJ

NJ â€”

o o

\C 00 00 \0

\0 i'

O O

o o

ON ÂŁ*

O^ ^ CO 00

^- Ln -^J **

NO LP -^ nÂŁ)

â€” U1

vO 00

00 ^

vO -C* vO LO

PI PI PI pÂ»

â– â€” UJ \0 â€”

O ^ w O

VÂŁ| w- \0 ,â€”

w O w o

*ÂŁ ^O nO sO

NO NO

w^

-63-

Let us now comment on the resuJ.ts. ''^irst of aJJ., the results for
DNl and ^N2 do not show anv sl?inif leant arivanta>^e Co either fom of the
nuitlplier estlTTate. This is an option onlv for Newton's method or a
method such as Al ,A? which approximates a full Hessian matrix. Methods
'^l through r)2 do not have the aptlon of using (2.7e^ since thev do not
approximate ''^. â–  .-

The results for Method. Al on Problems "4 and HSlll could he
improved hv reducing p, therebv improving the conditioning of the
matrices i\K '''e -chose â€” p ^ 10 because It is difficult to know an
oDtimai value of p _a priori, z '5ven then It was necessarv to Increase p
for Problem HS104, .where we used p = 100. The larger p is made, the
worse the results bfecoine", "because' of increased iil-conditionlng. Thus
the sensitivity of -Method Al bo -the choice of p is a definite drawback
of the method, Powell' s. Method A2. seems, from these results, to be
preferable. However we note^ithat no proof of local convergence has yet
been given for Method A2. ' '

The partial Rroyden methods R1-B4 are quite successful, but they
do seem inferior to Method A2 on the larger problems. The fact that
thev are one-step 0-superllnearly convergent does not seem to be a
use the ^'PG9, update. '''e should mention, however, that Method A2
theoretically has a one-step o-superllnear rate of convergence en
Problems HSl^n and HSlll, since ^J^ is positive definite for these
problems (and only these). The partial '^royden methods have the
advantage that they maintain an approximation matrix with smaller
dimension than that used by 'fethod A2.

The two-sided proiected Hessian methods C1-C8 compare favorably

with Method A2. Thev have the advantage that the di-nenslon of ( "^i, } is
further reduced. Methods ni-r)2 clearlv suffer from the extra iÂ»radient
evaluations required. ^gthods Cl-C^ ail have ahout the same
performance as each other. The same is true of Methods '^l-'^'i.

The parameter values n = 1 and v = O.ni for Methods Ci-CS were
successful for these test prohlems.. There vrouJ.d prohahiy never he a
need for a different choice of v, since the oniv purpose of this

1 3

Online LibraryJorge NocedalProjected Hessian updating algorithms for nonlinearly constrained optimization → online text (page 3 of 4)