Herbert Wesley Rogers.

Some empirical tests in vocational selection online

. (page 1 of 4)
Online LibraryHerbert Wesley RogersSome empirical tests in vocational selection → online text (page 1 of 4)
Font size
QR-code for this ebook






Submitted in partial fu'tillmenr of the requirements for the
degree - t Doctor of Philosophy, in the Faculty of
sophy, Columbia University





APRIL, 1922




Submitted in partial fulfillment of the requirements for the

degree of Doctor of Philosophy, in the Faculty of

Philosophy, Columbia University




No. 49

APRIL, 1922



1. The Problem. 5

2. History. 6

A. Tests for typists. 6

B. Primitive methods in vocational selection. 7

C. Traditional methods in vocational selection. 8

D. Modern methods in vocational selection. 8

a. Vocational miniature.

b. Vocational sample.

c. Vocational analogy.

d. Trade tests.

e. Miscellaneous empirical tests.

3. Procedure. 15

A. Materials employed. 15

B. Individuals serving as subjects. 15

a. Group 1.

b. Groups 2 and 3.

c. Age, education and experience of groups 1, 2, and 3.

C. Methods of obtaining measures of abilities. 17

a. Stenography.

b. Grammar.

c. Typewriting.

I. Group 1.

II. Description of the work of groups 2 and 3.

III. Group 2.

IV. Group 3.

D. Some remarks on the procedure. 21

E. Administering the tests. 22

a. General procedure.

b. Instructions to the subjects.

c. Records.

4. Results and how obtained. 24

A. Scores in the tests. 24

B. Treatment of errors. 25

C. Methods of combining the measures. 25

D. Correlations. 26

a. Stenography and grammar.

I. Method used.

II. Correlations.

b. Typewriting.

I. Method used.

II. Correlations.

E. Discussion of correlations. 27

a. Significant tests for typewriting and significant

tests for stenography and grammar.

b. Typewriting, group 1.

c. Typewriting, groups 2 and 3.

d. Typewriting, groups 1, 2, and 3.

F. Partial correlations. 29

5. Applicability of the results. 30
Appendix. 41

1. The supervisor as a judge of abilities. 41

2. Averages of the scores in the tests arranged according

to age and education. 42

3. Sample of an army trade test. 44

4. Bibliography. 45


1. The Problem.

The aim of this investigation is to test out in practice
the method of empirical vocational tests and to discover, if
possible, significant tests for specific commercial functions.

The type of work selected for correlation with the tests
is typewriting. Data in stenography and grammar, obtained
with one of the groups of subjects, have been utilized and in-
cluded in the investigation.


2. History.

A. Tests for typists.

Lahy (32)* reports an experiment made with eleven typ-
ists as subjects; six women who had had from two to four
years practice, and five men who had had from two to eight
years practice, in typing. Measures of the following func-
tions or processes were obtained; muscular sensibility of the
two hands, speed of auditory reactions, memory for numbers
and sentences, attention, and several higher processes such
as imagination, abstraction, judgment and reasoning. In
comparing the three good and the three mediocre women sub-
jects it was found that the former posessed a finer threshold
of tactile and muscular discrimination, a tendency of the
hands to become equally strong, a better memory, fewer er-
rors of attention, but showed longer auditory reaction times.
In comparing the men subjects the same distinctions were
found. This is an interesting and valuable contribution in
that it stimulates work along these lines and blazes the trail,
but it has not been worked out extensively enough to be prac-
tically applicable.

Lough (35) reports graphical curves showing the rela-
tion between the time required to perform a letter substitu-
tion test, "habit formation test," measured after repeated
trials 1 with the test, and ability in stenography and typewrit-
ing for a group of high school students. The curves show
a good degree of correspondence between performances in the
test and judgments of ability in stenography and typewriting.
To quote Loughf "It is evident that if we give these pupils
a test in habit formation first, we can determine what pupils
are capable of doing good work in typewriting, and we can
save some of the pupils a great amount of wear and tear, by
simply allowing them to take some other work in place of
typewriting. They are not adapted to the motor responses
which typewriting calls for. By means of this experiment it
would be possible to select those who are likely to succeed
in typewriting and to give a vocation to those who would not
succeed." If this test could accomplish all that Lough claims
for it it would indeed be a boon to the human race but un-
fortunately, the necessary scientific data concerning the ex-
periment are not given. There is only one test used and the

*Refers to Bibliography. fp. 94.


investigator who would build up a system of vocational guid-
ance or selection for such a type of work on the ability to per-
form any one such test, when the original data are inconclu-
sive, would be incautious indeed. However, a good deal of
credit is due Lough for his pioneer effort.

Link (33) gives a series of tests to detect ability in type-
writing and a series of tests to detect ability in stenography.
The tests are, for typing letter substitution, Trabue com-
pletion, context reading, spelling and typing ; for stenography
context reading, mixed letters and numbers, substitution,
Trabue completion, spelling, grammar, dictation and typing.

Link states* " relevant tests were given to two senior

classes of over three hundred girls and boys in a commercial
high school, to twenty-six pupils of two business schools, to
a group of twenty-two office typists, to another group of nine-
teen stenographers, to over four hundred candidates for posi-
tions as typists and stenographers the tests selected

on the basis of these experiments are those which showed the
highest and most consistent agreement with the abilities
of those examined." These are broad statements, made in the
absence of any norms, methods used, correlations, or any re-
liable data whatsoever, which should be presented if "consist-
ent agreement" is to be proved.

B. Primitive methods in vocational selection.

In primitive magic there is the naive idea that by simply
expressing a wish forcibly enough the wish will be material-
ized ; thus if the parents of an Indian boy wished their son to be
tall, they would toss him in a blanket. In medieval clair-
voyance there was and is still a belief that there are signs
which reveal an individual's character. The sources of these
signs are principally to be found in the zodiac, palm of the
hand, tea leaves, cards, dreams, crystal gazing, etc. In the
nineteenth century there was a spot located in the brain for
the function of speech and later all parts of the body were
found to be localized in the brain. Phrenology was a distortion
of this fact. Signs disconnected with the individual's body
are now given up and internal and personal external signs
take their place. In phrenology there was no comparison of
skulls, but individuals were taken at random who had some
peculiarity in the formation of their skulls; for instance, if

*p. 422-3 and chapter 8.


an individual had a bump anywhere on his head and was
also known to possess some peculiarity such as stinginess,
then that bump was labelled stinginess. When it became
known that there were no scientific criteria for phrenology,
the skull was discarded and the face was taken up, and the
pseudo-science of physiognomy was born. Physiognomy main-
tains that there is something in the features which is signif-
icant of inner hidden traits. This is true in so far as the
characteristic is an expression of a quality of character. But
in vocational psychology we are mainly interested in young
people whose characteristics have not been well developed or
well stamped upon their faces.

C. Traditional methods in vocational selection.

In the selection of employees the employer, for the most
part, has been content to form a ready estimate of a candi-
date's ability by looking him over, or else has accepted him on
the authority of someone who has recommended him; or
simpler still he tries the candidate at the task and, sink or
swim fashion, he promotes him or discharges him according as
he succeeds or fails in the work assigned. None of these
methods of measuring a man's value will bear criticism. The
last is always costly and time consuming; the second is un-
certain, depending on the degree of confidence that one has
in the recommender, his ability to evaluate and his motives;
as to the first there may be some men who are geniuses at pick-
ing men of worth at sight ; they seem to have an unerring in-
sight which prompts them to their choice, yet even in such
cases, it appears that one does not know the signs of value
intuitively but the ability to discriminate has been acquired
only through long and costly experience.

D. Modern methods in vocational selection.
a. Vocational miniature.

In the vocational miniature, the work is reproduced on a
small scale apparatus which duplicates the actual situation
which the worker faces while engaged at his task. Typical of
this method is an experiment of McComas of Princeton, in
which he attempted to detect good and bad telephone operat-
ors. He constructed an actual switchboard on a small scale.
The operator, to be tested, made connections at the board
which were timed on a kymograph in an adjoining room. This


species of "reaction and co-ordination" time showed the in-
terval between the appearance of a light over a call connection
and the moment that the operator "plugged in," and also be-
tween the moment that a number was called and the appropri-
ate connection made. Fifty-one trials were made by each of
the nine operators tested and the average time of each subject
was compared with the average ranking of the subjects by the
two supervisors. The test detected the two best and two out
of the three poorest operators. The correlation of these rank-
ings with the rankings of the supervisors was .6250 with a
probable error of .14. This method has the advantage of con-
creteness and apparent relevance but there is little actual as-
surance that the ideas and feelings aroused correspond with
the processes aroused in the actual work.

b. Vocational sample.

The method of vocational sampling is closely related to
the method of miniatures. It simply consists of taking an
actual piece of the work to be performed and sampling the
candidate's ability by the degree of his success in the trial.
Thus, in testing applicants for jobs as filing clerks, they may
be given a number of cards to be arranged in order according
to date, number, alphabetical position, address, etc. In cer-
tain cases specimens of work have been devised or taken into
the psychological laboratory and the worker watched more
carefully and measured more exactly. Unfortunately this is
a method which cannot be applied to very many kinds of work,
for care must be taken to make the tests representative of all
the work. At its best this method detects only the presence of
ability and is not a means of gauging potential ability.

c. Vocational analogy.

This method consists of an attempt to create, in the lab-
oratory, a situation which arouses and exercises the same
mental functions which the actual job arouses and exercises.
The material is new, but the attitude and endeavor of the
worker are intended to be the same. Munsterberg stated
(37) that, by this method, sea captains could be selected who
could be relied upon to avoid accidents. The test material
consisted of twenty-four cards, the size of playing cards, on
which are printed four rows of capital letters namely, A E
U in irregular repetition and interspersed with a few other
letters. The person to be tested had to distribute these twen-
ty-four cards as quickly as possible into four piles according


to the numerical preponderance of the four capital letters.
The time taken to sort the cards and the number of errors
made is taken as the index of ability to safely guide a ship's
course. The author of this test admits that there was no
correlation scientifically determined between the test and the
actual ability to guide a ship's course with safety. The insu-
perable difficulty with this method is that there is no way of
ascertaining that the same mental functions which are called
into action on the job are really being exercised and tested
in the experiment. This is a highly dangerous method and
has done much to put vocational psychology into disrepute
in the eyes of men of practical business affairs.
d. trade tests.

The method of trade tests is that of putting specific prob-
lems, which are germane to the work to be performed on the
job, to the applicant for a job. The answers to the questions
give an index of his experience and ability on the job. This
idea has been made use of in the past in the examinations
of candidates for positions by some concerns and chiefly by
some civil service commissions. (Appendix 3 for sample of
trade test.)

A good trade test should differentiate between various
grades of skill, should produce uniform results in various sec-
tions of the country and in the hands of individuals of widely
different characteristics and should consume the least amount
of time and energy consistent with satisfactory results.
While there are all degrees of trade ability among the mem-
bers of any trade, the method of the trade tests has classified
all of the members of any trade into a few groups, usually
four, such as novice, apprentice, journeyman, and expert. The
novice is a man who has no trade ability whatever, or at least
none that could not be paralleled by practically any intelligent
man. An apprentice has acquired some of the elements of the
trade but is not sufficiently skilled to be entrusted with any
important task. The journeyman is qualified to perform al-
most any work done by members of the trade. An expert can
perform quickly and with superior skill any work done by men
in the trade. It is essential that the trade test differentiate
between the journeyman and the apprentice, and the appren-
tice and the novice. Trade tests devised to make this classi-
fication are of three kinds: oral, picture, and performance.
The oral tests are most generally used because they are of low


cost and may be applied to a large number of men in a compar-
atively short time without much equipment.

The following stages are characteristic of the usual meth-
od followed in obtaining a trade test.

1. Investigation and collection of trade data. An investi-
gation is made into the conditions of the trade to determine
the following facts, (a) The feasibility of a test in the trade,
(in one instance it was found that the trade of gunsmith was
not a recognized trade, though there were gun repairers),
(b) The elements which require and permit of testing to de-
termine whether men can be graded in the trade according to
degrees of skill, (in some trades it has been found that the
trade required simply the performance of a single set of oper-
ations and there were no gradations among the members of
the trade), (c) The kinds of tests that can be used, (some
trades such as truck driving are mainly matters of skill and
for them performance tests are better than oral or written
tests.) Other trades such as interior wiring and power plant
operation are mainly matters of knowledge. For these trades
oral and picture tests are better. After having discovered
by inquiry that the trade is a recognized trade and can be
tested, all the information necessary is collected in the field
from all available sources, such as experts of the trade, trade
union officials, literature of the trade, trade school author-
ities, employers and the like.

2. Compiling the questions. As a result of the informa-
tion collected, a number of questions, usually forty to sixty are
compiled, each of which calls for an answer which shows
knowledge of the trade. The experience of the formulators
of trade tests has shown that a good question meets the fol-
lowing requirements, (a) It must be in the language of the
trade, (b) It must be a unit, complete in itself and requiring
no explanation, (c) It is not a chance question that could be
answered by a good guess, (d) It must be as short as possi-
ble and must be capable of being answered by a very short
answer, (e) It must not be ambiguous.

3. Preliminary sampling. After the large number of
questions originally formulated has been sifted down by the
application of the requirements listed in the preceding para-
graph the questions are used in a preliminary sampling on a
number of tradesmen whose answers indicate the merit of the
different questions and their grades of difficulty. In this


sampling tradesmen from different shops or plants are tried,
in order to guard against specialized methods or modes of ex-
pression confined to a single locality.

4. Revision and formulation. The preliminary sampling
affords a means of checking on the following points, (a) Is
the test applicable to trade conditions? (b) Does the test rep-
resent good trade practice? (c) In what way can parts be
profitably modified, supplemented or eliminated? (d) Does
the test represent the whole range of the trade from the novice
to the expert? (e) Is it a representative sampling of the whole
range of trade processes? In the light of the answers to these
questions the test is revised and then formulated.

5. Final sampling. Final sampling is made by testing
a number of men, usually twenty, who are known to be typical
members of each group (novice, apprentice, journeyman, ex-
pert.) Among the novices tested are usually some highly in-
telligent and mature men of good general knowledge but no
trade ability. Statistical treatment of the results and of the
answers to each question enables the determination to be made
of a relative value of each individual question and the selection
that makes a proper balance.

6. Evaluation. If a trade test is good, a known expert
when tested, is able to answer all, or nearly all, the questions
correctly; a journeyman is able to answer the majority; an
apprentice a smaller part ; and a novice practically none. This
does not mean that each question should be answered correct-
ly by all the experts, a majority of the journeymen, some ap-
prentices and no novices. There are a few questions which
show this result. Other types of questions, however, are more
common. Some show a distinct line of cleavage between the
novice and the apprentice. Novices fail, but apprentices,
journeymen and experts alike answer correctly. There are
likewise questions which are answered correctly by nearly
all the journeymen and experts, but only a few apprentices and
also questions that only an expert can answer. Each type of
question has its value in a good test. The main requirement
is that the tendency of the curve of distribution of the scores
should be upward. A question that is answered correctly by
more journeymen than experts or more apprentices than
journeymen is undesirable and is at once discarded. A proper
balance is made of the others.

7. Calibration. As each question is allowed a number of


points, usually four, it becomes necessary to determine how
many points should indicate an expert, how many a journey-
man, etc. This is accomplished by noting how many points
were scored by the known experts and the known journeymen
when they were tested. Ordinarily the expert scores higher
than the journeyman and the journeyman higher than the
apprentice. It frequently happens that a few journeymen
score as high as the lowest of the experts and a few appren-
tices as high as the lowest of the journeymen. There are, con-
sequently, certain overlappings between the classes. In cal-
ibrating, the object is to draw the dividing line between class-
es so that the overlapping shall be as small as possible. When
these dividing lines, or critical scores have been established
the test is ready to be applied.

Picture tests are made in practically the same way as the
oral tests. The peculiar characteristic of picture tests is
that the questions making up the tests relate to illustrations
of trade tools and appliances.

Performance tests are fundamentally the same as the
method of the vocational sample discussed earlier.

Certain important difficulties of the method of the trade
tests may be mentioned. One is due to the fact that trade
terms are very fluent, for instance the word "plug" probably
means a score of different things in many trades, and prob-
ably no single one of these twenty "plugs" is universally
known by that name. Localisms exist in various parts of the
country such as "come along" in the linesman's trade in New
York City. Foreigners, who compose no insignificant part
of the total number of tradesmen, have a different nomencla-
ture and one which is not usually a literal translation of the
English. There is a difference in trade procedure in different
parts of the country, carpenters trained in New York, Georgia
and California will answer the question "What is done
first in building a house" in very different ways and each
in a way which is right according to his training. People may
be coached, with ease, upon items of specific information an
individual with the barest trace of literacy can be taught to
answer the following written questions: "What is your
name?" "In what country were you born?" "In what state in
the Union is your domicile?" Coaching is quite as easy in other
specific information tests. Good trade tests determine simply
the presence of trade experience and the degree of skill actu-


ally possessed. They are not prophetic tests which indicate
probable aptitudes for trades which the person tested has
never followed.

e. Miscellaneous empirical tests.

Finally, there are cases in whch tests having vocational
significance have been sought by purely haphazard and em-
pirical ways. These may be designated as empirical vocation-
al tests. This method consists of selecting at random a num-
ber of tests and obtaining measures of large groups of work-
ers in any vocation and finding out if there are any tests which
have a high positive correlation with the actual work. This
is a strictly empirical method which is not based upon any
apriori postulates as are most of the other methods. In the
homely phrase of Carlyle, these tests seem to be "significant
of much/'


3. Procedure.

A. Materials employed.

Nine tests were used, selected at random, from the group
of tests which Woodworth and Wells (53) have standardized
for the American Psychological Association. In groups 2
and 3 it would have been unwise to have attempted to use more
than this number since it took approximately thirty minutes
to test a subject. Under commercial conditions this was a
great amount of time for the concern to give, and also for the
subjects to give since they were losing their bonus for the
time spent on the tests. The tests used are opposites, verb-
object, agent-action, action-agent, color naming, mixed rela-
tions, hard directions, number checking and form substitu-
tion.* Of those tests in which more than one form is given by
Woodworth and Wells the form which was used in this experi-
ment is as, follows.

* Considerable time was spent by many subjects on the stimulus words
"love" in the opposites test, on "kiss" in the verb-object test, and on
"lungs" in the agent-action test. It was evident, that for some subjects,
it was more difficult to respond to these words than to others of the list.
If the tests were to be given over again these words should be taken out
and others, of the same difficulty as the rest of the list, substituted for

1 3 4

Online LibraryHerbert Wesley RogersSome empirical tests in vocational selection → online text (page 1 of 4)