Albert Ando.

# Evaluation of an ad hoc procedure for estimating parameters of some linear models online

. (page 1 of 2)
Online LibraryAlbert AndoEvaluation of an ad hoc procedure for estimating parameters of some linear models → online text (page 1 of 2)
Font size LIBRARY

OF THE

MASSACHUSETTS INSTITUTE
OF TECHNOLOGY

coey I

Evaluation of an Ad Hoc Procedure for Estimating
Parameters of Some Linear Models

A. Ando and G.M. Kaufmati
94-64

MASS. ir^CT. Tc

OCT 30 1'

DEWEY LIDRA:

MASSACHUSE

50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETl

COPY I

Evaluation of an Ad Hoc Procedure for Estimating
Parameters of Some Linear Models
A. Ando and G.M. Kaufman.
94-64 j

MASS. INST. TcCH.

OCT 30 ^9P4

Authors are Associate Professor of Economics and
Finance, University of Pennsylvania and Assistant
Professor of Industrial Management, Massachusetts
Institute of Technology. The contribution of Ando
to this paper was supported by a grant from the
National Science Foundation. The authors acknowl-
edge the contribution of Mr. Bridger Mitchell of
Massachusetts Institute of Technology, who prepared
the computer program for calculations reported in
this paper.

Y\A

NOV 9 1965
,, , I. LlBRAKlE-S

Economists and other users of statistical methodology often posit a
probabilistic model of some real world phenomenon which has more unknown param-
eters than there are sample observations. In such cases it is usually impossible
to jointly estimate all parameters from the sample data. Even in those instances
where there are well established estimation procedures when the number of sample
observations n is larger than the number of parameters r, these methods are
generally inadequate when n < r^ as the necessary calculations cannot be carried
out. Furthermore, when n is only "slightly larger" than r, such estimates often
prove to be unreliable in more than one sense.

One particular example of such a problem that frequently occurs in analysis
of psychological and of economic data is this: the researcher posits a linear
regression model as defined in (2) below with r-1 independent variables but
only n < r observations on the dependent variable. Since the standard least
squares procedure cannot be applied, he may ask the following seemingly reason-
able question: "What subset of the r-1 independent variables should I select
for inclusion in a "new" model to which I can apply the standard least squares
procedure?"

Our purpose here is to demonstrate that one frequently used ad hoc method
for determining such a subset by ordering simple sample correlation coefficients
can be highly misleading. Any procedure which uses a given set of sample data
to both determine the structure of the model to be tested and to estimate param-
eters of this model is intuitively unsettling. Here we present tables which
quantitatively demonstrate how dangerous such ad hoc methods can be.

Ad Hoc Use of Simple Correlation Coefficients to Determine Model Structure

Consider an r-dimensional Independent Multinormal process defined as one

that generates independent r x 1 random vectors x
densities

~(1)

,x

(J)

4'^(2ikh) = (2

^)-ir e"^(3S-li)''^(2i-y.) |h|2

with identical

(1)

h is PDS

~(k)

We wish to estimate parameters of the conditional distribution of xj^ ' given
x^ ,...,x^ ' when neither ^ nor h is known with certainty from a set of n
vector sample observations x "^ ^x ,...^x^" . Alternatively, we may consider
an r-dimensional Normal Regression process defined as a process generating

independent scalar random variables according to the model

;p>,

'i + iiz A^^^i. ^^^""^

(2)

the X. s are known numbers which in general may vary from one observation to

the next, and the e.s are independent random variables with identical normal
' J

fj^(e|0, h) = (2n)"

(3)

Suppose now that we have n observations, and let

(1)

^2

.. . [po] > ... > |p [‚ÄĘ

3. Choose an integer r* < rank (X X).

4. Restructure the model, eliminating from consideration

relabelled variables x .,.,..., x . (Henceforth we call the
r*+l' ' r

initial model, "Model A" and the restructured model, "Model B".)

5. Assuming that Model B is an adequate approximation to Model A
use X X to estimate parameters of the conditional distribution
of X, given the values of relabelled variables X2,tT>u^oao x>Nrr¬Ľao¬ęo -J-Lnvooo
.n-iosr¬ę-vo vOMooor* in-vjooh* ^o¬ę^-*c^3^ >0(n-¬Ľoo
-¬ę-i -*
f>4 ^ -¬ę -¬ę _¬Ľ

J> r>j vO o in
>J ¬Ľ /> m M

O -< O vO

r^ -*‚ÄĘ N o

oo o f^ o> (*>
NJ r^ 4^ "M -H

>i -< -H -< r-<

‚Ė†St r-i -* ^ ~t M'-¬ę-|^-4

- H -*‚Ė† ^ u> vO

t>a r* -*‚Ė† -‚ÄĘ c^

.->< -H -^ ^ o

00 '-' -t i-t ^ (MiAOO'^O

(¬ę>a04oac> inOOrOO
'>J-¬Ľ - -4 r¬ĽfN4i 00 in (*\

ro rvj ^ ^ ^

vO CO CO O rS ^

O O 30 >J >o

^ ^ r^ ^ ^

>J M -H - I -1

-gp^‚ĄĘ¬ęr - ao oooo^f^in
>OoaoinfO r-'HoovO^

'Mrg,^^^,^ fOrgpg.-¬Ľin

.v - i >^r'it/\m.fl f^vOOr-in

O¬Ľr¬Ľ\0¬ę>O-* "^^JOr^in ng,n - in

ro fM -M

‚ÄĘ ‚ÄĘ ‚ÄĘ

00
rM

fM

rg in ro
r- o r^
fJ

fO
fM

fM

vO

in

rO

in fM
r- in
fM fM

‚ÄĘ ‚ÄĘ

‚ÄĘ

‚ÄĘ ‚ÄĘ

o>

OO
fM

t~- in

fM fM

‚ÄĘ ‚ÄĘ

r*^ fO

‚ÄĘ ‚ÄĘ

O fM 4-

-t ON r-

fO (M fM

u-l
PI

‚ÄĘ

‚ÄĘ

00

o

fM

‚ÄĘ

CO

‚ÄĘ

fO
fO

‚ÄĘ

CM C7>

or-
f^ rg
‚ÄĘ ‚ÄĘ

ON

in
rg

00 00

r-i in

‚ÄĘ ‚ÄĘ

(M

O ON

o r-
ro iM

-g o

^ CO

0¬Ľ 00 -<
fM O ON
m ro "g

S

O

o

‚ÄĘ

fvj
ro

‚ÄĘ

fM

o

in

fM

‚ÄĘ

-*‚ÄĘ
fM
fM

‚ÄĘ

00 ro
Oco

cr\ rM

fM

‚ÄĘ

ON fM
ro 00
,t fO

‚ÄĘ ‚ÄĘ

fO

‚ÄĘ

r~ ro
-. O

‚ÄĘ ‚ÄĘ

^ 00

‚ÄĘ ‚ÄĘ

00 fM O
- >J- (^
r- o r~
>t -i- 3- rg
-

in ^

in ^ On

o r- ^
-3- CO ro
‚ÄĘ ‚ÄĘ ‚ÄĘ

O

CO

‚ÄĘ

m O ro
o in r-t
-* fO CO

00
fM

‚ÄĘ

in

‚ÄĘ

>*‚ÄĘ
‚ÄĘ

00 fM O
O vO fM
fO rO rO
‚ÄĘ ‚ÄĘ ‚ÄĘ

in r-
in ^

in

fM

-4-
‚ÄĘ

r-t in

ON vO

‚ÄĘ ‚ÄĘ

O Oi

in OO
in >?

‚ÄĘ ‚ÄĘ

r- 00 r-

>¬Ľ‚ÄĘ O 00
4- >3- CO
‚ÄĘ ‚ÄĘ ‚ÄĘ

m

00

OO

o

00

‚ÄĘ

fM

in

in ¬ęo

o

in
o

,t CO
O fO

vO in

a a

i*>

‚Ė†M ^

iN in
vO in

a ‚ÄĘ

CO in ^
rH r- in
¬ęn ^ -t
a ‚ÄĘ e

O

in

o \r\ o
r- o -3-
ir> lA ^

vO

O

o

fM ¬ęo

OP- O ^ CO

'H o o o o
o o o o o

o ^ (> o in
rg m ro en iM
-1 o o o o
o o o o o

¬Ľ-‚ÄĘ O CO OS P^
sO rO in
o o
o o

‚ÄĘ ‚ÄĘ

f>j o p- in ^
r^ ^ o o o

o o o o o
‚ÄĘ ‚ÄĘ ‚ÄĘ ‚ÄĘ ‚ÄĘ

o OS 00 m 00
00 in f*\ fsj rg
o o o o o
o o o o o

‚ÄĘ ‚ÄĘ ‚ÄĘ ‚ÄĘ ‚ÄĘ

o

r-

o

‚ÄĘ

fM t^ in gj
^ o r- r-
rg - < ^ O O

o o o o o
o o o o o

-< rvj CT. ^ ^rt

en -1 o o o
o o o o o
o o o o o

O 00 O O 00
rg -H -< r-4 o
o o o o o
o o o o o

O vO *

: ^ rg f\i r-t -H

o o o o o

o o o o o

-t ro

1

Online LibraryAlbert AndoEvaluation of an ad hoc procedure for estimating parameters of some linear models → online text (page 1 of 2)