Educational psychology online

Bagley and Rugg ('16) made a study of twenty-three text-books,
pubHshed between 1865 and 1912, by comparing the amount of
space given to various topics in each book, by determining the
shift in emphasis in the course of this period of time as measured
by the space given to different topics, and by Hsting the topics and
names included in all, or in a certain fraction of these texts.

"(i) In so far as can be determined from the materials presented in
the text-books, elementary American history as taught in the 7th and 8th
grades has been and still is predominantly political and military history.

" (2) Within the past fifty years, the emphasis upon military affairs
as measured by the proportion of space devoted to wars has declined. In
general, battles and campaigns are treated less in detail than was formerly
the rule, while proportionately more space is devoted to the causes and
the results of the wars. The lessening emphasis upon details of the wars
is first noticed in some of the text-books published between 1881 and
1S88, and the tendency has been general and decided since that time.

" (3) The later books give a perceptibly heavier emphasis to the facts
of economic and industrial development than do the earlier books, al-
though pohtical development stiU constitutes the essential core of ele-
mentary historical instruction.

" (4) As regards the treatment of specific eras or epochs, the principal
increases in emphasis are to be noted in connection with: (a) the period



HISTORY 425

1783-1812 (especially in the treatment of the so-called 'critical period'
between the close of the Revolution and the adoption of the Constitu-
tion); (b) the non-military affairs of the period 1S12-1861; and (c) Euro-
pean events preceding and during the periods of discovery, exploration,
and settlement.

"(8) Numerous changes have taken place in the construction of ele-
mentary text-books in history during the past fifty years. The more
important of these are: (a) a movement toward a simpler 'st3de' with
larger emphasis upon clear statements of causal relationships: (b) the
introduction and development of the 'problem' as a method of teaching
history, and a consequent encouragement of 'judgment' as contrasted
with rote memory, — of rational as contrasted with verbatim mastery;
(c) a marked decline in the employment of imaginative pictures as illus-
trations and an increase in the use of pictures that represent sincere at-
tempts to portray actual conditions; (d) a marked decline in the use of
anecdotal materials; (e) a larger and wider use of maps." (Pp. 56 and
57.)

Horn ('17) conducted an investigation after the manner of
Bagley's magazine-newspaper method by checking through
twenty-seven recent books on current industrial, political and
social problems, in order to ascertain the facts, persons and dates
referred to and the frequency of reference. His general impression
of this inquiry is stated thus:

"This investigation has not attempted to answer the question as to
the complete content of the course of study in history. Neither does it
assert that the purpose of history is to throw light on modern social
problems, or that this is even one of the chief purposes of studying history.
Without regard to what the aims of teaching history are, this investiga-
tion has been carried on to examine into the implications of one par-
ticular assertion: namelj^ that history should render pupils more in-
telligent with regard to modern conditions, problems and activities. If
one assumes (i) that this is the function of history, (2) that the method of
research here followed is satisfactoiy, and (3) that sufficient data have
been collected, then there seems to be no escape from the conclusion that
the present elementary and high-school courses of study in history are
in very serious need of reconstruction." (Horn, '17, p. 171.)



CHAPTER XXII

MARKS AS MEASURES OF SCHOOL WORK

Importance of Marks. In order to determine the fruitfulness
or wastefulness of methods of learning and teaching school sub-
jects, it is necessary to evaluate the achievements of pupils as
accurately as possible. Furthermore, the successful operation of
a school demands an accounting of the work of its pupils.

Marks have been the universal measures of school work. So
many problems in the management of a school — credit, failure,
promotion, retardation, elimination, graduation, honors, recom-
mendations for positions, indeed the entire scholastic machinery
of a school — hinge upon the assignment of marks that it is highly
imperative to examine in detail the value, accuracy and reliabiUty
of marks as well as to ascertain the possibility of some sort of
standardization of marks.

Variations among Teachers and Schools in the Distribution of
Marks. The manner in which marks are distributed to pupils
varies enormously from teacher to teacher and from school to
school. No one realized the seriousness of the situation until
specific tabulations and comparisons were made.

Meyer published the distribution of the marks assigned by 40
dififerent professors at the University of Missouri to their students
during a period of five years as exhibited in Table 131.



426



MARKS AS MEASURES OF SCHOOL WORK 427



TABLE I _ I. After Meyer ('oS)

Distribution of the marks of 40 teachers in the University of Missouri for a
period of live years. The numbers are the percentages receiving the
various grades.

Total No.

Teachers A B C F of Students

Philosophy 55 33 10 2 623

Latin 1 52 42 6 o 130

Sociology 52 30 13 5 958

Mathematics 1 40 31 16 13 208

Economics 39 37 19 5 461

Greek 39 26 24 11 287

Latin II 36 40 19 5 577

French 36 29 25 10 295

Political Science 34 30 27 9 592

Mathematics II 32 29 23 15 14S

German 1 30 29 20 11 586

Psychology 1 30 36 24 10 907

German II 26 38 25 11 941

Elocution 20 61 19 o 917

Geology 22 48 22 8 293

History 1 14 53 27 6 779

Zoology 1 21 45 28 6 479

Psychology II 19 47 29 5 238

History of Art 25 40 30 5 685

Bacteriology 20 45 31 4 263

Freehand Drawing ... 18 47 25 10 506

Chemistry 1 23 40 31 6 205

English 1 21 41 30 8 964

Astronomy 13 49 33 5 225

History II 11 51 33 5 806

Zoology II 24 37 31 8 250

German III 22 37 28 13 44i

Chemistry II '9 48 43 o 21

Education 18 38 35 9 266

Mathematics III 19 36 26 19 182

Mathematics IV 25 29 36 10 380

Physiology 20 33 40 7 426

Anatomy 19 34 36 11 544

Mathematics V 16 34 35 15 209

Engineering 1 13 36 42 9 813

Mechanical Drawing. . 18 29 41 12 538

Mechanics 18 26 42 14 495

Engineering II 16 26 46 12 826

Chemistry III i n 60 28 1903

English II 9 28 35 28 1,098



428



EDUCATIONAL PSYCHOLOGY



I



Similar tabulations have been published for Harvard University
by Foster, Table 133 ; for the University of Wisconsin by Dearborn,
Table 134; and for Cornell University by Finkelstein. Finkelstein
('13) has shown the effect of the personal equation in the marks
assigned to the same students in a year course which was in charge
of one teacher in the first semester and of another in the second
semester.



TABLE 132. After Finkelstein



I St semester . .
2nd semester .



No. OF
Students

â–  263
257



Per Cent Receiving the Various Mares
0-39 40-44 45-49 50-54 55-59 60-64
•4 -4 2.3 4.5 1.9

— — 1.2 .8 1.5



13-7
12. 1



65-69
II. 8
12.8



No. OF
Students



I St semester, con . .
2nd semester, con. .



70-74 75-79 80-84 85-89 90-94 95-100 Exempt
263 16.7 15.6 20.2 6.4 5.7 .4 12.5

257 10.5 13.6 9.8 33.9 3.8 — 37.7



The instructor in the first semester exempted fr%m the final
examination 12.5% of the class, while the instructor in the second
semester exempted 37.7%. The latter obviously graded very
much higher than the former.



MARKS AS MEASURES OF SCHOOL WORK 429



TABLE 133. After Foster ('11).
Harvard College. Distribution of 8,969 grades. Elementary Courses

Group I A "To B% C% D% E% Abs.To Total

Astronomy 16 13 45 19 6 i 69

10 17 48 17 7 2 130

Botany 11 28 38 14 2 7 183

4 32 44 13 K 6 219
Chemistry 6 26 45 12 9 2 334

8 19 45 17 II o 319
Economics 10 18 37 25 7 3 531

7 19 ' 43 21 7 3 436

Engineering 11 15 31 2S 12 3 114

Engineering 10 13 27 21 21 9 121

139
129

English I 13 52 28 3 3 603

I II 51 32 3 3 564

Fine Arts 2 S3 45 1° 2 9 58

6 27 67 o o o 49
French 11 25 35 21 4 4 156

12 19 36 19 10 4 145

Geology 5 26 45 20 3 2 489

5 25 2s 28 2 7 85
Geology 2 28 48 10 7 5 122

4 20 43 24 7 2 108

German 7 21 31 26 11 4 259

6 14 32 27 17 2 293
Government 6 16 39 28 8 3 356

9 23 37 21 7 2 419
Greek 35 28 21 13 i 3 72

IS 36 34 7 5 3 61

History 7 20 44 21 5 2 347

7 24 42 20 6 2 380
Hygiene 18 29 ^^ 18 i o 87

8 23 48 14 4 3 139
Latin 17 25 41 10 7 o 143

15 27 41 5 10 2 128

Mathematics 18 24 18 31 11 o 85

14 22 31 23 II o 95

Philosophy 7 31 41 15 2 5 229

7 23 61 8 o K 215
Spanish 10 24 43 16 4 3 106

7 ^3 3& 33 8 2 119

Zoology 2 13 48 30 5 I 149

184

Averages 7 20 42 21 7 3 213



43°



EDUCATIONAL PSYCHOLOGY



TABLE 134

Percentages of grades assigned by 45 individual instructors in the University of
Wisconbin to freshmen and sophomores. After Dearborn ('10)



History :
I . .



3-
4-
5-
6.

7-



English:



13-
14-

15-
16.

17-
18.
19.



Mathematics:



23-
French:

24-
25-

26.

27-

28.
29.



Physics and Chemistry:

30

31



Latin:
32.
33-
34-



Ex.


G


F


P




X


No. OF












Cases


4-9


26.2


32.8


26.2


9.8


183


9


8


52.9


31.6


3"


2-5


193


3


4


22


38 -9


26.7


8.9


558


7


4


25.2


37-4


23-4


6.5


107


iC


7


52.4


28.6


2-3





42


9


I


39-4


27-3


18.2


6.0


33


6


3


27.4


30.8


23.6


II. 8


237


19-3


22.6


193


38.7





31


12.5


30


37-5


12


5


7-5


40


6.4


45


31.2


14


7


2.7


109


1.9


33-2


45-5


12


9


6.4


202


7-4


30


6


38.0


19





4.9


121


0-3


33


7


40


14


7


5-2


95


4.0


3«


8


32.6


14


3


10.2


49


16.7


22


8


36.0


17


5


7.0


114


7-3


39


6


37-5


13


5


2.0


96


16.9


50


6


195


II


7


1-3


77


II. 4


42


9


27. 2


14


3


4.2


70


8.9


53


9


21.8


7


6


7.6


78


6.1


25


5


235


37


8


71


98


16.6


27.8


28.5


14.2


12.9


302


24.1


25


15-7


19.4


15-7


108


12. 1


17.9


19-3


27.4


23-3


223


21.2


43-8


22.5


7-5


5


So


17-5


43-8


27s


7


5


3-7


80


22.3


35-7


21.4


12


5


8.0


112


15-5


32


27. 2


19


4


5-8


103


145


24.2


24.2


27


4


9.6


62


23-9


35-4


24.6


13


8


2.3


130


27.9


37-8


21. 1


10.8


2.4


204


21. 1


35


24.6


15-6


3-8


289


16. 1


46


20. 7


9.2


8.0


87


II. 7


46.2


26


16





119


26


.1


47


.6


15-9


7


•4


2.8


107



MARKS AS MEASURES OF SCHOOL WORK



431



jerman:

35 26.3

36 12.0

37 34-3

38 11-4

39 ^7-3

40 17-9

41 14-7

42 12.3

43 29.0

44 21 .6

45 22.4



n



20



J



10







34-2


21.9


12.3


5


2


114


49.1


24.1


II . I


3


7


108


40.3


19.4


2.9


2


9


67


34-4


27.9


22.9


3


2


bi


37-3


29-3


6.6


9


3


75


35


26


17. 1


4





123


29-5


33-5


18.9


3


I


95


27.4


27.4


21 .9


10


9


73


30.1


22.6


14


4


3


93


42.0


21.6


12. 5


2


27


88


39-6


20.7


^S-S


I


7


5«



hi




70 75 80 85 90 95 100

Fig. 85.— Distribution of all grades in a high school. After Gray ( '13, p. 66).

These tables agree in >howing extremely wide differences among
teachers in the manner of giving marks. In the tabulation for the
University of Missouri, one professor assigned the grade A to 55%
of his students, and the grade F to only 2%, while another profes-



432



EDUCATIONAL PSYCHOLOGY



sor assigned the grade A to only i% of his students and the grade
F to 28%. At Harvard, one professor gave the grade A to 35%
and the grade E to 1%, while another professor gave A to 1% and
E to 32% of his students.

The situation in high schools is substantially the same. Gray
tabulated the marks assigned by all the teachers in eight high



9*



S»



!•■



m



/If



1



76 75 80 85 90 95 100

Fig. 86.— Distribution of all grades in another high school. After Gray ('13, p. 67).

schools. The distributions of two of these schools are shown in
Figures 85 and 86. The one grades high and the other grades low.
In one school the great mass of the pupils receive 85 to 100; in the
other they receive 85 to 70.'

Table 135 shows the distribution of the grades by the different
teachers in a high school of about 150 pupils.



MARKS AS MEASURES OF SCHOOL WORK 43;:



TABLE 13s

Distribution of 8,490 grades by the different teachers in a high school during
four years. (From a private report by Superintendent J. F. Waddell,
Evansville, Wisconsin.)

— 74 75-80 8t-86 87-92 93-100

English (1913-1914, 1914-1915). . 33% 39% 13% 12% 3%

English (1915-1916) 15 27 27 25 6

Latin and German 5 15 13 ^;^ 34

Mathematics (1913-19 14) 22 31 18 21 8

Mathematics (1914-1915) 23 27 24 13 13

Mathematics (1915-1916) 5 12 15 28 40

History (1913-1914) 11 25 22 33 9

History (1914-1915; 1915-191O).. 10 18 25 30 17

Science (1913-1914) 11 36 25 26 2

Science (1914-1915; 1915-191O) . . 10 33 24 19 14'

Domestic Science o 12 27 51 10

If
Distribution of grades for the year after the above tabulation was made known

to the teachers. Extreme variations are considerably reduced.

English (1916-1917) 6% 32% 27% 27% 6%

Latin and German (191O-1917) ..9 25 22 29 15 •

Mathematics (19 1 6-19 1 7) 5 20 28 27 20

History (1916-1917) 2 17 37 34 10

Science (1916-1917) 5 26 33 .22 14

Domestic Science (1916-191 7) ... o 10 40 41 9

Variation among Teachers in the Evaluation of the Same School
Products. A more direct and crucial method of examining the
variations of teachers' marks than the tabulation of the grades as
distributed by different teachers is to measure experimeatally the
differences in the values assigned by different teachers to the same
pieces of work.

Starch and Elliott ('12 and '13) made a series of investigations
in which two final examination papers in first-year-high-school
English were graded by 142 English teachers in as many high
schools, one final examination paper in geometry was graded by
118 teachers of mathematics, and one final examination paper in
American history was graded by 70 teachers of history. The
variations in these marks are shown in Figures 87, 88, 89 and 90.
The differences are astounding; the marks for any given paper run
practically over the entire range of the percentage scale ordinarily
used. The marks of the first English paper run all the way from



434 EDUCATIONAL PSYCHOLOGY

64 to 98, of the second English paper from 50 to 98, of the geometry
paper from 2 8 to 92 , and of the history paper from 43 to 90.



• • •

• • ••

• • • • •

• ••••••

• ••••••••

• • •••••••••

•• ••••••••••••



65 70 75 80 85 90 95

Fig. 87. — Distribution of the marks assigned by 142 English teachers to a
final examination paper in high-school freshman English. After Starch and
Elliott ('12).



•••• ••••



•50 60 65 70 75 80 85 90 95

Fig. 88. — Distribution of the works assigned by 142 english teachers to an-
other final examination paper in high-school freshman English. After Starch
andEmott('i2).



• • • • • •

• •••• ••••

• ••• ••••••• •••• •••• * •

• • '« •••»•••••••• ••••••••• ••■• •

••••••• ••••••••••••••••••••••••••• •••••• •

28 53 55 60 65 TO To S ^ i)

Fig. 89. — Distribution of marks assigned by 114 mathematics teachers to a
final examination paper in geometry. After Starch and Elliott ( '13).



40 50 60 70 . 80 90

Fig. qo. — Distribution of marks assigned by 70 history teachers to a final
examination paper in American history. After Starch and Elliott ('13).



MARKS AS MEASURES OF SCHOOL WORK 435

This investigation has established two conclusions: first, that
teachers differ enormously in evaluating the same pieces of work
in terms of the ordinary percentage scale; and second, that they
differ as much in one subject as in another. They disagree as
much in evaluating a paper in mathematics as in Enghsh or his-
tory. Apparently mathematical papers are not marked with
mathematical precision any more than any other papers are.

The author made a further investigation by having ten final
papers in freshman English in the University of Wisconsin graded
by ten instructors of freshman English. The marks are shown in
the following table:









TABLE


136.


After Starch ('


13)






Marks


assigned by


ten


instructors to ten final examination papers in


English.
























Coeffi-










Instructors












cient


Pa-


















AVER-


Mean


OF Varia-


pers 1


2


3


4


5


6


7


8


9


10 AGE


Var.


BILITV


1 8S


86


88


85


75


80


88


87


85


87 84.6


2.8


.034


2 77


80


87


80


62


82


82


87


85


87 80.0


4.6


.057


3 74


78


78


75


69


84


91


83


79


80 79.1


4.4


.056


4 65


65


62


20


26


60


55


68


55


50 52.6


12.3


.233


5 68


82


78


82


64


88


85


86


78


80 79.1


5.7


.070


6 94


87


93


87


83


77


89


88


88


89 87.5


3.2


.036


7 88


90


95


87


79


85


96


91


87


89 88.7


2.6


.029


8 80


84


73


79


72


83


85


91


77


76 80.0


4.6


.058


9 70


70


68


50


44


65


75


81


79


79 68.1


9.1


.118


10 93


92


85


92


81


83


92


89


84


85 87.6


4.0


.045



Av. 79.4 81.4 79.8 73.7 65.5 78.7 83.8 85.1 79.7 80.2 78.7 5.3 .074

The variations shown in this table are practically as large as
those found in the previous inquiry. It was thought that the wide
range of marks shown in the first study might be due to the fact
that the teachers were in different schools. However, Table 136
shows that teachers in the same department differ almost as much.
Less extensive results obtained by having various members of a
department grade the same paper show that as much variation
exists in other subjects as in English.

Causes of Variation. Why do teachers differ so much in estimat-
ing the worth of a given product and in the distribution of marks
to groups of pupils? Four possible factors may be mentioned:
(i) Differences in the standard of severity or leniency in different
schools; (2) differences in the standard of severity or leniency of
different teachers; (3) differences in credit or penalty assigned by
different teachers to any given fact or error in a piece of work; and
(4) minuteness of the discrimination between successive steps of



436 EDUCATIONAL PSYCHOLOGY

merit or quality in a given scale of qualities. How potent is each
factor in producing the total variation in evaluating a given paper?

The first thought that occurs in regard to the wide range of
marks for the same papers as shown in Figures 87 to 90, was that
it must be due to the fact that these teachers were situated in
different schools with different standards and ideals. It turns out,
however, that factor one is relatively insignificant. If we compare
the mean variation of the marks of the ten English papers assigned
by ten instructors in the same department with the mean variation
of the marks of the two English papers assigned by teachers in
different schools we can determine the part played by factor one.
The mean variation of the former set of marks is 5.3 and of the
latter is 5.4. Hence the mean variation of the marks assigned by
teachers in the same department is only o.i less than the mean
variation of marks assigned by teachers in different schools.

The potency of factor two may be ascertained from the data in
Table 136. The general average of all the grades assigned by the
ten instructors to the ten papers is 78.7. If we compare the aver-
age of all the marks given by any one instructor with the general
average we obtain a measure of his particular standard of severity
or leniency. Thus instructor 5 graded on the average 13.2 points
lower and instructor 8 graded 6.4 points higher than the general
average of all the teachers. If now we raise or lower each instruct-
or's grades by as many points as the average of his grades is below
or above the general average, we find that the mean variation of
these weighted marks is 4.3. This mean variation is only i.o
point smaller than the mean variation of the original unweighted
marks. Hence factor two accounts for a relatively small share of
the total variation. Factors three and four must then account for
the remaining mean variation of 4.3. The strength of factor four
can be determined experimentally by having the same teachers
re-grade their own papers without knowledge of their former
marks. The author carried out such an experiment and obtained
the results exhibited in Table 137.



MARKS AS MEASURES OF SCHOOL WORK



437



>

^^

W 5 tr-

tj « a

a'"'

S
w




Of''^>t^'-'f^^»oO^


O
4




OMTl-P)0"OCOror^

00 o 00 00 00 00 c^ r^ ooo


lO

00


H


O ro (N to lOOO 00 fO ro f^

t^ ooo r^r^t^oooo ooo


o

IN

CO


Z; < W
< > t


Q


lO lO lO

0\ ro • r^ CN C\ • lO • M

M


lO

o


Q
2


OJ^rOM O MOOO O \r>
t^ t^ J^OO OO MD lOOO vO >o


t^


H


lO K) IT)

0> O • lOOO O ■ 1-1 • o

I^ On t^OO t^ t^ <N Os <N m3




w < ta

W > H
J Oi z

o w o




loo O ^ M •-• O r^


00


a
•z.


lOO 00 O O VO lO to
t^OO 00 t^ t^OO O t^


OO


H


O O 00 "^1- t^ lo looo
t^co OO r^ r^oo O >0


O

o


a S z




-^rO'^IMOlN'NfOOfO


N


Q
Z


O rotoO <N r^OO lo^
vo t^ r^ OO 00 00 >o r^O




H


VO O t^OO <N 0\ (N fO lO t^

lo t^ t^oo O 00 OO lo f^o




3 > fe:


Q


»o

MO O O lOOO 00 o J-~


co


Q
Z

(S


M r^ r^ OnO ro C> 1^ O
loo O J^OO O t^OO O


!0

4-


H


OM MrOMMMOfO
roo O t^oo r^ t~~ C>00


CO

d


>

o
o

a > u

>• M "

u

W


Q


0-*rO<NiOi-i-^0<NCO


fO


Q
Z


0\ ro ro <N 00 O OOO O O
t-^oo Ov OOO r^oo 00 t^oo


00




lO I - O O rOOO COOO CO fO
oO CO O O^CO t^ O>00 r^oo


lO

lO

oo


, <

Oh ^

H




cs '^roi-i M ioOnio Tto


^


Q

y.


t~- O O <-ooo looo lOO
CO oo oO O^oo 00 t^oo 00 oo


00


H


loo roO\-:l-rO-+rOOv<^
oo r - 00 00 OO OOO Ooo O


00

o

00








<



438 EDUCATIONAL PSYCHOLOGY

The mean variation of these marks, comparing the first with the
second for each paper, is 2.2 points. A part of this variation,
however, is due to the slight shift in standard on the part of each
teacher from one grading to the next. By applying the same
process of weighting explained in connection with factor two, the
mean variation drops from 2.2 to 1.75 points. Hence 1.75 repre-
sents the amount of variation contributed by factor four to the
total mean variation and by subtraction we find that factor three
contributes 2.55 points. The four factors therefore contribute
the following amounts:

Factor one 10 points

Factor two i . 00 "

Factor three 2 . 55 "

Factor four i • 75 "

Total mean variation 5-4° "

It is obvious then that factors three and four are the most im-
portant ones in producing the large differences of values assigned
by teachers to a given piece of school work.

How Large Should the Units of a Marking Scale Be? The
answer to this question depends primarily upon the fineness of the
discriminations of successive degrees of quality in terms of the
scale used, and, secondarily, upon the convenience of using a gi^•cn
scale of marks. The smallness of distinguishable shades of quality
of anything can be determined by ascertaining the amount of
difference in terms of a given scale that can be discriminated in
the long run by the judges.

A general principle that has been followed commonly in psycho-