Copyright
Jorge Nocedal.

Projected Hessian updating algorithms for nonlinearly constrained optimization online

. (page 3 of 4)
Online LibraryJorge NocedalProjected Hessian updating algorithms for nonlinearly constrained optimization → online text (page 3 of 4)
Font size
QR-code for this ebook


-44-

Theorem 4.3: Suppose "pdate Rule 1 Is user! in Aisorithm 4.1.
Choose a value v "> in condition (4.3). Then

\/re(0,n,3e>0, 5>0,ri>0, such that if He II _< e ,

II B -H^ll ,^ < 5, and n is used to define condition (4.3), then

n e^+^W < nie^.^ll , k ? 1 , (4.21)

i.e. , xj^ > x^ at least at a two-step 0-linear rate.

Proof: We prove the result for the PSB and OFP cases, leaving the

BFGS case as an exercise. Let a,a,,a2 be the constants from Lemma ^.3,

_ o

and define y^, M as in Corollary 4.2. Let a^ = II Mil ^a^. Let 3 be such

that



XII < B 11X11^



for any n X n mtrix X. Let ic = 2liH;'^ll , 5 = 1/ (211 H^^^!!) , so that, bv the
Banach lemma,



X-H,^ll < 5 => II X ^11 < K . (4.22)



Let E,,C, be as in Theorem 4.1, using this definition of 0, 5 > 0, n > to be
chosen small enough that



-45-



e < e ^/C^



e < £9/0^^



6 < 5/(23)



C^(e + 2B6 ) < r



Mil - {209(1 + n)C, e + niiG^ii } < 1



'1



•• ■ 4C9a + n)C,e ,

(2a, 6 +09) ( : ^ + n(i +-] IIG^Il) < 5 .



(4.23)



Such a choice of e ,6 ,n is clearly possible. Now let 11 e^!l < e,
II B -H^ll X, < 5. We will establish by induction that



(i) IIB-H*II^ < 26 , .1 > .

(ii) II eyi < C^lle^.^ll , 1^1.

(iii) lle^^ll < rllej_-Jl , j > 1 • . (4.24)

(i V ) If_ the update formula is used at iteration j ,

then IIB^^^-H^IIm < [(1 - aO ^)^^- + a ^y jl" B .-H^ll ^ + a9Y ^

else B :j+i = B . , i > .



Note that (iii) is the desired result.

For j = 0, (i) is true by assumption and (ii), (iii) are vacuous.
For (iv), we need only verify the inequality assuming the update
formila is used to obtain B^. Because II e^H < E < t -^ and H B^ II < < ,
Theorem 4.1 applies (with k = 1), giving



-46-



e^ll

Note that (1 -ct6^ )'■'-< 1 - a ^. Now from (4.24) (iv), we have



\.+r"*"M* [(1 -ote2_)i/2 +ci^Y,,.]llB^_-H^il +ajy^^_ ,



where, proceeding as before to apply Corollary 4.2, and usin as i + " .



S, II

'^i



Thus, using either (4. la) or (4.1b) for the definition of Sj^, and with
U|^ defined by (4.12), we obtain



(jj^ •»■ as i + » .



Now from Theorem 4.1(iii), for i large enough,



■52-



X 1 1 1 u



and hence bv ( 4. 2 4) (i 1) , (ill') ,



Thus



^k = li r"^ n as i > » ,

i "^k.-l" ,



I.e. , {Ci wJ NJ —


> >


tsj —




c




wi wi


ui ui \yi Ln


Ui Ui Ui Ui


O* OS CT 0^


ON


i> X>






^ ^










-s






- -


C^ ^ O^ o^


ON o^ o^ a^


^ ^ ^ ^


■^


NJ 00






3 —


U) LJ UJ UJ
H- O^ NJ NJ


J> £^ X> i>

VD — — UJ


Ul U1 Wl UI
N) U) W —


UI +
l'


o

— rsj

U b

P! PI

1 1


3

ISJ

3


3

C




N3 00

1 1


00 O vTi —

c=: p: m r=3
1 t 1 1


O — CT^ C

p] p: m p:
1 1 1 1


00 O -o £-

PI p] n m
1 1 1 1




CC


.- — O r^J


NJ < fSJ


o o o o


^


i— ho








Ui LP


^ ^J -^ -^


-J -J ^ ^


i^ ui i-n £>


ON


X> £-






^ ^










^








CO 00 00 03
Ui k^ ui m


00 03 OC 00
Ui ui i»n ui


On On O^ Ui
www

J> *^ i> UJ

J> UJ U) -^




(SJ 00

O O


L.J

II
rsj


0=

r-
p:

2






rO N3 PO LJ

n p: m n


yi ri :^ T=i


O NO NC Ul

P3 PI m p:


o


ON J>
P] PI


ISJ




VJ5 «


— ,- — —


— — — —


— — — ^

J> ^ £-. ^-'


O


X> f>






^


w w w w


WW w^




Ui J>


Ui Ui ^ IJ1


i-n ui UI Ln


ON CT* ON ON


Ln vO


I> £>






'^








. -^


."'^ ■''^.






— vo


c> c^ cr o^


0^ O^ O O^


•^ -^ -J -J


OS O ■


A-n "— „


::.


-'




J> oJ


(^ L-i Lo ro


U> UJ LJ K)


i-n LD Ui Ln


£> 00


o o


3
II


3
II


C

i

-a

wJ




— 'J


!> J> ^ Ln


Ui m Un 03


00 00 ^ —


^ 00


[SJ *»J




O -J

1 1

- vO


\0 ^O vO ^

n rr: pj n
1 1 1 1


— — ^ o

p; p: p: P!
1 1 1 1


-J X> O UJ

— PI p: PI
1 1 1 1

NJ N> SJ O


PI p]
1 1

n£> no


£- SO

PI P5

: 1

N3 J>















z


^ ^


CO 00 00 OS


03 X 00 -^


ON OS ON OS


CO - J


U> Lf>






^O ^£1


O '^ ^ sO


uo o ^ o


^ '^ -^ -O


NO 00


\0 i>


^


-3


2:


00 00


'^ *J ^J -vl


■^ - J -^ 00


Ui Ui ui Ln


kJ


tsJ L-J


w




\0 SJ 00 ^

P3 pj M rn


i' !T> O ISJ

PI PI P3 PI


(jJ W ^- tsj

PI P] P5 P3


ro o

m PI


vC —




VO H-


\0 ^ vO ^o


i£3 ^ sC —


•^D ■■^ \D ■-0


so NO


55






w ^


Sj NJ NJ —


ro ro NJ H-


^- — tsj NJ


ISJ


Ui Ui








uj UJ uj n:


uj uj uj ro


N3 |NO W UJ


— +
V

sC


UJ ISJ
— OS

c o

ON VO
PI PI
1 1


D
1

3
II

(SJ


T3
C

2

i

c
o






*^ vO "^ vO


— — — o

— l^ »— Ul


o o <

vO 00 NJ Ln




ro ON

PI W
1 1

O —


O Ln UJ Ui

PI m w rr
1 1 t 1

o o o o


Ln cjv i* J>

p: PI PI PI
t 1 t 1
^ ^ ^ ^


1^ -^ *» Ln

P! PI m p:
1 1 1 1

^j2 \C ^ "—




\o o


^ V.O ^C O


O v£> ^ ^C


ro NJ ro u)


O Ui


Ul Ul




-3

p:




— Isj


o o o o


C C O O


1^ UJ UJ x>


— 0^


«; uj


II

O




00 \o


00 09 00 00


00 00 03 03


— " ^ I^


vO i^


O O
















^

^


5i




fJN i-


— _ ^ ^


— — O —


— ON NJ Ui


— —


OS £^






Ln CT^ Ui UJ
P5 PI PI PI


•^ UJ ON -C*
PI PJ P5 PI


O \0 C3N x>

m p: p: p:


O NO

m p:


Ul UJ

p: p:




^O vO


>X) \0 — vC
w w o ^


vO vO ►— v£

w W O w


00 *0 \Ci \£>


NO NO


CC
















^ -o


00

3
II


-3

PI

EH

o






o o ^O o


o — o —


1^ Ui 1J1 i_n


— —






— Z 5 ^


— ISJ — ro


ON On O* ON


N> NJ


NJ —




o o


\C 00 00 \0


\0 i'


O O


o o




ON £*


O^ ^ CO 00


^- Ln -^J **


NO LP -^ n£)


— U1


vO 00






00 ^


vO -C* vO LO

PI PI PI p»


■— UJ \0 —

O ^ w O


V£| w- \0 ,—

w O w o


*£ ^O nO sO


NO NO


w^









-63-

Let us now comment on the resuJ.ts. ''^irst of aJJ., the results for
DNl and ^N2 do not show anv sl?inif leant arivanta>^e Co either fom of the
nuitlplier estlTTate. This is an option onlv for Newton's method or a
method such as Al ,A? which approximates a full Hessian matrix. Methods
'^l through r)2 do not have the aptlon of using (2.7e^ since thev do not
approximate ''^. ■ .-

The results for Method. Al on Problems "4 and HSlll could he
improved hv reducing p, therebv improving the conditioning of the
matrices i\K '''e -chose — p ^ 10 because It is difficult to know an
oDtimai value of p _a priori, z '5ven then It was necessarv to Increase p
for Problem HS104, .where we used p = 100. The larger p is made, the
worse the results bfecoine", "because' of increased iil-conditionlng. Thus
the sensitivity of -Method Al bo -the choice of p is a definite drawback
of the method, Powell' s. Method A2. seems, from these results, to be
preferable. However we note^ithat no proof of local convergence has yet
been given for Method A2. ' '

The partial Rroyden methods R1-B4 are quite successful, but they
do seem inferior to Method A2 on the larger problems. The fact that
thev are one-step 0-superllnearly convergent does not seem to be a
significant advantage, compared with the disadvantage that thev do not
use the ^'PG9, update. '''e should mention, however, that Method A2
theoretically has a one-step o-superllnear rate of convergence en
Problems HSl^n and HSlll, since ^J^ is positive definite for these
problems (and only these). The partial '^royden methods have the
advantage that they maintain an approximation matrix with smaller
dimension than that used by 'fethod A2.

The two-sided proiected Hessian methods C1-C8 compare favorably



with Method A2. Thev have the advantage that the di-nenslon of ( "^i, } is
further reduced. Methods ni-r)2 clearlv suffer from the extra i»radient
evaluations required. ^gthods Cl-C^ ail have ahout the same
performance as each other. The same is true of Methods '^l-'^'i.

The parameter values n = 1 and v = O.ni for Methods Ci-CS were
successful for these test prohlems.. There vrouJ.d prohahiy never he a
need for a different choice of v, since the oniv purpose of this


1 3

Online LibraryJorge NocedalProjected Hessian updating algorithms for nonlinearly constrained optimization → online text (page 3 of 4)