oO 


A 





I: 
6 
“4 
ie) 
5 
iS) 
6 
G 
WS) 
6 
1S) 
6 
6 
Ny 
> 
i] 
6 
6 
rR) 
3 

19 
P| 
K) 
6 
ay 
tN) 








VOL. VIII, NO. 1 JANUARY, 1930 
Research Bulletin 
OF THE 


National Education Association 


6S666SS68S 





ISBSGaSaae aoasaoaaaaag 





THE QUESTIONNAIRE 


Have you dispatched a carefully prepared questionnaire, ask- 
ing for significant data? 


Has this questionnaire gone to a group of persons who ordi- 
narily treat you with courtesy and respect? 


Has your questionnaire been thrown into the waste basket 
with the others—good, bad, and indifferent—which roam the coun- 
try unrestrained? 


Or 


Have you neglected pressing duties, and burned midnight oil 
while you answered questionnaires? 


Have you anathematized questionnaire-makers and all their 
works? 


Have you purchased a large waste basket and resolved to an- 
swer no more questionnaires, question-blanks, check lists, in- 
quiries, or data sheets? 


If you have done any or all of these things—this Bulletin is 
for you. 


b 


(s] 


———_-_--—_- 





SSsGoREeseas 


SSS 


266666 








F 


12) 


J) 


Flalaizizt 








7} 





PUBLISHED BY THE RESEARCH DIVISION 


OF THE NATIONAL EDUCATION ASSOCIATION 


1201 SIXTEENTH STREET NORTHWEST, WASHINGTON, D. C. 


Entered as second-class matter February 10. 1923. at the Post Office at 
Washington, D. C.. under Act of August 24, 1912. Acceptance for 
mailing at special rate of postage provided for in Section 1103, 
Act of October 3, 1917, authdérized February 10, 1923. 


Baur 


Fh 


OBaaG 








Research Bulletin of the National Education 
Association 


Published five times each year in January, March, May, September, and Novembe: 
by the Research Division of the National Educatien Association of the United State. 
President, E. Rutu Pyrtie Secretary, J. W. Crasrri 


The payment of the $5 membership fee of the National Education Associatio: 
entitles one to receive the Research Bulletin, the Journal, the Annual Volume of Ad 
dresses and Proceedings, and certain other publications of the National Education Asso 
ciation for one year. One dollar of each $5 membership fee is for a year’s subscription 
to the Research Bulletin. The Research Bulletin may be subscribed for separately at 
subscription price of $1 per year. Copies may be purchased for 25 cents, and in 
quantities at a reduced rate. Those entitled to receive the Research Bulletin regularl, 
are asked to report at once any change of address, giving old as well as new address 








Entered as second-class matter February 10, 1923, at the Post Office at Washington, D. C., u 
Act of August 24, 1912. Acceptance for mailing at special rate of postage provided for in Se 





1103, Act of October 3, 1917, authorized February 10, 1923. 

Director of Research: John K. Norton Associate Director: Margaret Alltucker Norton 
Assistant Directors: Frank W. Hubbard, William G. Carr 

Executive and Editorial Offices - - - 1201 Sixteenth Street N. W., Washington, D. ( 








How to Secure Additional Copies 


Additional copies of this issue of the Research Bulletin may be obtained at the 
following prices, postpaid if remittance accompanies order: 


Single copies of this issue 25¢ 

2 to 9 copies—10% reduction 

10 to 99 copies—25% reduction 

100 or more copies—33 1/3°% reduction. 


Orders for Research Bulletins not accompanied by remittance are subject to trans- 
portation charges. Make checks payable to National Education Association. 


Subscriptions to Research Bulletins 


The Research Bulletin may be received regularly through a special subscription at 
$1 per year. Only those holding the $5 membership (see statement at top of this 
page) receive the Research Bulletin without such subscription. Many of the early 
issues of the Research Bulletin are already out of print. The special subscription at $1 
per year offers an opportunity to maintain a complete file of future Research Bulletins. 


Reproduction of Material in Research Bulletins 


Electro-mat-slide service: The National Education Association maintains a special 
service to aid those reproducing material appearing in its publications. Send orders or 
requests for further information to the Division of Publications. 

Tables, charts, and any other material appearing in Research Bulletins may be 
reproduced for the advancement of education. No authority other than this notice is 
needed previous to such reproduction. For further information on this subject or others 
affecting the work of the Division, address the Research Division, National Education 
Association, 1201 Sixteenth Street N. W., Washington, D. C. 








[2] 




















. 


PT RT pede awe 





CONTENTS 


PAGE 

Foreword ....... 4 
ED eine pie a csit'in 634i 6 A's 5 
Questionnaires—To Be or Not To Be.... 9 
The Case Against the Questionnaire. 9 
The Case For the Questionnaire... 1! 
Better Preparation of Questionnaires.... Siioe ace eee err 14 
Preliminary Considerations ............. 14 
Preparing and Sending the Questionnaire... 18 
Reporting Questionnaire Results....... per | 25 
The Questionnaire Problem From the Viewpoint of the Superintendent of Schools 28 
Evidence Is Assembled .... 28 
The Evidence and What It Shows.... 8 

A Proposed Plan For the Cooperative Regulation of Questionnaires 38 
Some Sources of Information...................... 44 
References on the Use of Questionnaires in Educational Research +7 
Forms Used in Collecting Data 49 

LIST OF TABLES 

Number of Questionnaires Reported By Cooperating School Systems (Table 1). 29 
Who Sends Questionnaires to the Offices of Superintendents of Schools (Table 2) 30 
The Fields of Education In Which Questionnaires Seek Information (Table 3) 31 


Month of Issuance of Questionnaires Sent to Offices of Superintendents of Schools 
EE a iateks Ciecca nee Vacs bas Uivd SOON RMR os ERM a TS OR CiNe Cameo a ELE dWialees 31 

Distribution of 267 Questionnaires Circulated in 1927-28 According to Number of Items of 
Information Requested (Table 5) ............. . paonne 32 


Number of Duplicating Questions in 267 Questionnaires Asking a Total of 13,070 Ques 


IN Sag. 5 cc acs Cahn 00's SU Red MER GATK oo 0 desc nude bbaenat 33 
Percent of Reply Received by Various Classes of Questionnaire Circulators (Table 7) 34 
Percent of Reply Received as Related to Month of Issuance of Questionnaires (Table 8 34 
Percent of Reply as Related to Length of Questionnaire (Table 9)......... cifeaie ie ace 
Uses Made of 171 Questionnaire Studies Begun During 1927-28 and Completed by March, 

ES os sha she dead ea Coed ame hivdor gck's : aoe i id 35 

LIST OF FIGURES 
Questionnaires and Poor Administration (Figure I).. 6 
Questionnaires and Good Administration (Figure II).... Nn eee 7 
Illustration of a Master Tabulation (Figure III).......... or ee “3 27 
39 


An Informal Rating Plan for Questionnaires (Figure IV)... Pree 








FOREWORD 


subject of inquiry is timely. Second, discrimination is exercised as to how it is 
investigated. Third, results are presented in a form likely to affect practice con 
structively. This issue of the Bulletin abundantly fulfills each of these criteria. 


\ YHREE characteristics distinguish fruitful educational investigation. First, the 


That the questionnaire is widely used in educational investigation is obvious. 
That its use has troubled many educators both in schools and colleges is equally clear. 
That more heat than light has been generated by some discussions of the “questionnaire 
problem” is testified by references cited in the bibliography. An investigation of the 
problem is therefore timely. 


Discrimination has been used in selecting proper methods of studying this difficult 
problem. Enough data have been presented, but not too much. Both the sins as wel! 
as the virtues of this “wayward, but indispensable” tool of science are laid bare. 


Finally, a constructive program is proposed. Those who would ‘“‘do something” 
about questionnaires need not hesitate as to what to do if they will carefully study 
the material made available herein. It offers a plan which is both sound and workable. 


One warning is desirable. The entire Bulletin should be read before final con- 
clusions are drawn. Material presented in certain sections reveals the questionnaire 
at its worst. Facts in other sections bring out the contribution it has made to educa- 
tion. The significance of certain paragraphs will be changed if they are taken from 
their context. If the central finding were to be stated, it would take some such word- 
ing as this: 


The abuses which have grown out of the circulation of questionnaires can be corrected by 
voluntary, cooperative regulation. -~For those who receive questionnaires this involves the 
inauguration of plans whereby worthy inquiries may be selected from the rest and answered 
with the least expenditure of energy. For those who send questionnaires, it calls for planning 
to the ends that no unnecessary nor improperly prepared inquiries may be circulated. It is 
important that these means of correction be taken. Otherwise one of the methods of investiga- 
tion which has contributed substantially to educational advance will be cut off at the source. 


This Bulletin is recommended to those who would exert a constructive influence 
in the regulation of the questionnaire problem. It will be of interest to superintend- 
ents of schools and their staffs, college specialists, graduate students of education, and 
others who directly or indirectly come in contact with questionnaire investigation. 


J. W. Crasrrer, Secretary, 


National Education Association. 

















[4] 





ye 











192; 
men 





Introduction 


ORACE MANN in 1847 sent a ten 
page “Circular” asking certain teachers 
in Massachusetts a series of questions. 

It began as follows: 


I desire to obtain the opinion of teachers who are 
both scientific and practical on a subject of great 
importance to the cause of popular education.’ 


He little realized that in sending this and 
subsequent Circulars to “persons whose ele- 
vated character, and whose extended personal 
acquaintance with the subject matter on which 
they testify, place them above denial, cavil, or 
suspicion” he was creating a precedent later to 
be followed by so many as to reach the propor- 
tion of a “problem.” 

The inception of this Bulletin—Recent years 
have witnessed a rapid growth in the use of the 
questionnaire as a means of educational investi- 
gation. Figures presented in a later section in- 
dicate that several hundred different question- 
naires are annually circulated among the of- 
fices of superintendents of schools in this coun- 
try, and that some individual school systems re- 
ceive more than a hundred each year. 

The work of answering questionnaires has 
come to be regarded as an unjustifiable burden 
by some. Magazine articles anathematizing the 
questionnaire-makers and all their works have 
appeared. Some school systems have taken 
blanket action against answering all such in- 
quiries. 

The members of the Department of Super- 
intendence of the National Education Asso- 
ciation have repeatedly, as individuals and as 
an organization, taken notice of the “question- 
naire problem.’’ When the original proposal to 
establish the Educational Research Service was 
made seven years ago by R. G. Jones,” a prin- 
cipal motive was to reduce the number of ques- 
tionnaires by establishing a “clearing house’ 
which would make many of them unnecessary. 
Official notice was again taken of this problem 
in the resolutions of the 1929 meeting of the 
Department of Superintendence. 

Questionnaires have become so unpopular 
that many who send them avoid the word. In- 





stead they issue “inquiry forms,” “check-lists,” 
and “data sheets” accompanied by apologetic 
notes. ; 

These conditions stimulated the Research 
Division to study the whole problem. The col- 
lection of material bearing upon it began sev- 
eral years ago and an intensive study of some 
phases of the question has been under way for 
the last two years. 

The purpose of this Bulletin—This issue of 
the Research Bulletin presents material perti- 
nent to the proper solution of the problem. The 
aim is to answer these questions: 


. Do questionnaires serve useful purposes, and 
if so, what are they. (See pages 11-3.) 

- How may questionnaires be better prepared? 
(See pages 14-24.) 

. How may the number of questionnaires be re- 
duced? (See pages 14-7, and 38-43.) 

. How may the results of questionnaires be made 
more useful? (See pages 25-7.) 

. What is the size and character of the problem 
of dealing with questionnaires as it is en- 
countered by superintendents of schools? (See 
pages 28-37.) 

. What should be the attitude and practice of 
superintendents of schools in dealing with 
questionnaires? (See pages 13, 38-43.) 

- How may the various interested agencies co- 
operate in an intelligent regulation of the ques- 
tionnaire problem? (See pages 38-43.) 


General conclusions—The outcome of in- 
vestigations of the questionnaire problem which 
this Bulletin presents may be epitomized in 
these words: 

The practice of circulating questionnaires in 
the field of education has undoubtedly been 
abused. The time of busy people is sometimes 
wasted by time-consuming questionnaires deal- 
ing with inconsequential topics, worded so as 
to lead to worthless replies, and circulated by 
untrained and inexperienced individuals, lack- 
ing in facilities for summarizing and dissemi- 
nating any worthwhile information which they 
may obtain. 

From these facts, however, one should not 
draw the conclusion that the questionnaire 
should be abolished. There are competent in- 


' Life and Works of Horace Mann. Lothrop, Lee and Shepherd Co., Boston, Mass., Vol. IV, page 151. 

2 Jones, Robinson G. ‘‘A Program for the Coming Year."" Addresses and Proceedings of the National Education Assox iation, 
1923, pages 907 to 909. For information concerning the value and cost of the Educational Research Service address the Depart- 
ment of Superintendence, National Education Association, 1201 16th St. N. W., Washington, D. C. 


[$] 





I SS 





Figure I 


QUESTIONNAIRES «xd POOR ADMINISTRATION 


Is this the way you handle questionnaires ? 


Superintendent of Schools: These 
questionnaires are 6 nuisance. 
1 have no time to answer a 
them . 




















Does this happen when you want school facts ? 


Superintendent of Schools: How can 
| advise my board when there are no facts ? 
Director of Research . Sorry ty 
“AL 


sir, but_ you know we never an- 
swer questionnaires and in- ia | Va ~? \ 
vestigators have no other way ay 

to get such eeommion 9 





































_-o 





: 
> 
Le 
t 


Figure II 





~ QUESTIONNAIRES GOOD ADMINISTRATION 





Do you handle questionnaires this way ? 





Superintendent of Schools : Have this ad 
questionnaire evaluated and if? it X 
deserves attention see that the inform 

ation requested is furnished. 


— 
——\ 
a \ 
\ 
- 
. f 
NX #/ 








Is this what happens when you want school facts ? | 











Superintendent of Schools: Excellent, ! 
This is just the information I need in 
advising my board. 

Director of Research : The facts came. 
from @ questionnaire 
study in which we 
recently cooperated. 














vestigators who issue carefully prepared ques- 
tionnaires on vital educational questions and 
make the results generally available in usable 
form. 

A broad and unbiased study of the question- 
naire movement as a whole leads to the con- 
clusion that this method of investigation, in 
spite of its abuse, has yielded far more on the 
credit than on the debit side of educational ad- 
vance. Much of the significant knowledge we 
now have in the field of education has come 
from the questionnaire in the hands of qualified 
investigators, and for some time to come it will 
undoubtedly constitute a legitimate means of 
educational investigation and research. Super- 
intendents of schools and their staffs, professors 
of education, and graduate students of educa- 
tion, to name those who most often originate 
questionnaires, find them indispensable, and 
they will continue to circulate them. This be- 
ing the case, what should be the attitude of one 
who wishes to exercise an enlightened influence 
on this problem? 

If he is one who circulates questionnaires he 
will examine material presented in, or cited by, 








this Bulletin? which will help him to prepare 
questionnaires well, which will guard him fro.) 
sending unnecessary questionnaires, and w!| 
will aid him in making the replies to questi: 
naires most useful to himself and others. 

If he is one who receives questionnaires 
will examine the material presented in, or ci: 
by this Bulletin which will aid him in 
tinguishing between worthy and unwort! 
questionnaires, and to set up a simple, continy 
ing provision for economically answering those 
which are worthy. 

Finally, whoever he is, school administrato: 
research worker, or teacher, he will recognize 
that it is the duty of every one to do what ! 
can to advance educational knowledge, whethe: 
his contribution be large or small, direct or in 
direct. He will strive to make the energy ex 
pended in meeting this responsibility count for 
most. He will not waste the time of others }) 
circulating poorly prepared, unnecessary in- 
quiries, on inconsequential topics, nor will he 
allow his own time to be wasted by answering 
such inquiries. 


1 When references are not given in complete form in the footnotes, they will be found fully cited in the bibliography on pages 


47-9, 


| 8] 


lh AA gM Py 








\W 


an 


scl 


to 


sec 
to} 
na 


est 
*tin 


pa 
an 
ize 


tio 
tif 
toc 
Ea 


is ¢ 
litt 
Re 
up 
rec 
cer 
of 


per 





| 
. 


LO Ne a 5S teat RNAS 


~ 


Questionnaires—To Be or Not To Be? 


There is ample evidence that the question- 
naire has fallen into disfavor with many. Some 
would even go so far as to issue a blanket in- 
terdict against this method of investigation. 
What justification is there for such an atti- 
tude? Let us examine the evidence both against 
and in favor of this method of inquiry. 


The Case Against The Questionnaire 


These are the objections frequently raised 
against this method of assembling data: 

1. The results of most questionnaires make 
little if any contribution to educational ad- 
vance. First-rate educational investigations do 
not employ this method. Conclusions based on 
questionnaire blanks are seldom final. 

2. The respondents to a questionnaire may 
not be a random sampling of those addressed. 
Selection may invalidate the representative- 
ness of data, particularly if the percent of reply 
is small. 

3. Respondents to a questionnaire may con- 
sciously or unconsciously modify their replies 
to give the investigator what he wants. 

4. Carelessness in reporting, difficulty of 
securing information requested, and other fac- 
tors favor inaccuracy in replies to question- 
naires. The degree of this inaccuracy cannot be 
estimated. 

5. The questionnaire method is indefensibly 

‘time-consuming particularly for the respondent. 

6. Questionnaires are frequently poorly pre- 
pared, ask for information already available, 
and the results are often not properly summar- 
ized and distributed. 

These objections deserve serious considera- 
tion. Are they valid to the degree which jus- 
tifies the abolition of the questionnaire as a 
tool of the serious educational investigator? 
Each will be considered in turn. 

Much effort for inconclusive results—There 
is evidence that some questionnaires contribute 
little if anything to educational advance. The 
Research Division made an intensive follow- 
up of 267 questionnaires on a variety of topics 
received during 1927-1928 by the offices of 
certain superintendents of schools. No report 
of any kind could be obtained as to what hap- 
pened to 64 of the 267. A considerable per- 





1 See page 36-7 of this Bulletin. 


centage of the questionnaires for which reports 
could be obtained were not tabulated or used 
in any way. Less than half of them eventuated 
in either a mimeographed or printed report 
which could be circulated among respondents. 
It appears that many questionnaires make little 
contribution, even to the one circulating them. 
These facts, however, do not justify the con- 
clusion that no first-rate contributions result 
from questionnaires. Conclusive evidence to the 
contrary is presented later, 

The objection that questionnaire investiga- 
tion seldom conclusively answers the question 
at issue is not a serious one. In modern science 
no answer to any question is final. Many sup- 
posedly final answers to problems in the field 
of physics, a most exact science, have been 
revised in the last decade, and the final answers 
are still being sought. 

Selection of replies—The danger that the re 
plies to questionnaires may represent a selected 
group is a real one. There are undoubtedly 
situations where this factor invalidates conclu- 
sions. ‘his danger is not an insuperable one, 
however, for these reasons: 

a. A well prepared and properly motivated 
questionnaire, with careful follow-up, may 
secure such a high percent of reply that the ad- 
dition of the missing replies could not signifi- 
cantly modify data actually received. For ex- 
ample, 67 of 68 cities over 100,000 in popula- 
tion replied to the extensive salary data blank 
sent out by the Research Division in 1928- 
1929. A reply from the missing city could have 
only slightly affected the general results. 

b. Objective appraisals of this factor such 
as that of Toops, cited in the bibliography on 
page 49, suggest that the danger of a selected 
response has been overestimated. Until evi- 
dence to the contrary is forthcoming, it seems 
reasonable to conclude that this factor is prob- 
ably not decisive in affecting the results of the 
typical questionnaire, particularly if a reply of 
more than 50 percent is received. One can never 
be sure on this point, however, and unless a 
reply of practically 100 percent is obtained it 
is well to heed Kelley’s words of caution: 


. . . lack of demonstrable fairness in the sample 
is probably the greatest shortcoming of the ques- 


[9] 





tionnaire. The user of the questionnaire should 
never lose sight of it.” 

Desired answers given—Concerning the ob- 
jection that respondents tend to give answers 
desired by the interrogator, there are doubtless 
situations in which this is a factor of signifi- 
cance, particularly when subjective opinions or 
attitudes are called for and leading questions 
are used, It is unlikely that this objection ap- 
plies to most well-prepared questionnaires for 
these reasons: 

a. The investigator often does not, and 
properly should not desire responses, other than 
those which present the truth, and even if he 
does have his private opinions, they are not 
known to the respondents. 

b. Most questionnaires are largely made up 
of questions of fact, answerable with yes, or 
no, or other objective reply, and reduce the 
number of questions calling for a subjective 
reply to a minimum. Hence, the probability 
that answers are modified so as to suit the in- 
vestigator is remote. 

Data fundamentally inaccurate—The ob- 
jection that the questionnaire inherently en- 
courages careless reporting, and therefore se- 
cures inaccurate data, is a serious one. It has 
also been pointed out that respondents are 
either unable or unwilling to give accurate an- 
swers to certain types of questions calling for 
personal or confidential data. 

These are unanswerable indictments if small 
differences are significant and extreme nicety in 
original data is essential. There are many edu- 
cational questions, however, in which an ap- 
proximately correct measure of central tend- 
ency, or a general indication of trends, is suffi- 
cient for the purpose. The sometimes over- 
worked, but none the less valid statistical prin- 
ciple that chance errors tend to neutralize each 
other, particularly in influencing general aver- 
ages, also applies here. 

An alert investigator will discover means of 
checking at least portions of his data. For ex- 
ample, rechecks of data secured by the salary 
inquiry of the National Education Association, 
made in connection with intensive local salary 
investigations, indicate these data to be sufh- 
ciently exact so that not only the ranks, but the 
absolute position of individual cities may be 
designated with reasonable confidence. 





1 ’ man L. Scientific Method, page 81. 


Kelley, Tru 
? See page 33 of this Bulletin. 


Education is still far enough from bein: 
exact science so that data assembled by 
method inherently involves a degree of e: 
often unmeasured, if recognized at all. I{ 
questionnaire is ruled out on this basis p: 
cally all other methods of educational iny 
gation deserve a similar fate. 

Time-consuming—The objection that qu 
tionnaires are indefensibly time-consuming 
particularly of the respondent’s time, is valid 
in too large a percent of cases. This, howe, 
is not an inherent objection. An intelligent 
lection of only worthy questionnaires for atten 
tion, and proper organization of such time an 
facilities as one has to give to them, will dis 
courage the circulation of inquiries which are 
unnecessarily time-consuming. 

Poorly prepared and handled—That unn 
essary, poorly prepared questionnaires are ci: 
culated by people who lack facilities for sum 
marizing and disseminating results may be con- 
cluded from facts presented elsewhere. The in 
vestigation upon which this Bulletin is based 
however, did not reveal a serious amount of 
overlapping among questionnaires.” It is true 
that a considerable number on a common sub 
ject may be issued each year. An analysis of 
these inquiries, however, reveals that they fre 
quently deal with different phases of the same 
general topic, and the amount of overlapping 
of specific questions is small. It appears that the 
amount of duplication represented by ques 
tionnaire investigations has been over-empha 
sized. 

An appraisal of the objections—A review ot 
the foregoing objections to questionnaires re- 
veals none that justify blanket condemnation 
of this method of assembling data. Most of 
them consist of criticisms of the abuse of, 
rather than the right use of, the questionnaire. 

The admitted fact that many questionnaires 
contribute little or nothing, loses much of its 
significance when weighed with equally in- 
dubitable evidence that some questionnaires 
contribute a great deal. The situation is not 
unique. Any form of educational investigation 
works much ore for a little gold. The lessons 
from this fact, as it affects questionnaires, 
should be—start with better ore, and refine it 
by better methods. Practical means for doing 
this are suggested subsequently. 


[10] 





ques 
stan: 
suffi 
the : 
that 
ques 
gato 
exac 
meth 
gests 

T 
obvi 
naire 
are f 
peop 
no gi 
of ti 
naire 
solut 
ature 
veals 
argul 
fies ; 
aboli 
catiot 
Le 
tionn 
quiry 
comp 
But + 
naire 
T 
“B 
many 
vield 
tribut 


Cc 
Chicag 


Other important objections to the question- 
naire method—that a selected group of re- 
spondents may reply, and that it inherently en- 
courages inaccurate reporting—lose their po- 
tency except when applied to poor execution 
in using this method of investigation. If it ap- 
pears, because of the character of the informa- 
tion needed and the composition of the groups 
which can supply it, that a selected reply would 
be likely to be received, then the questionnaire 
probably should not be issued. If, however, a 
questionnaire is issued under such circum- 
stances, follow-ups should be sent out until a 
sufficient percent of reply is received so that 
the factor of selection is minimized. Realizing 
that a degree of inaccuracy may characterize 
questionnaire replies, the competent investi- 
gator will either not use this method when 
exact data are requisite, or will provide a 
method of validating results, as Koos? sug- 
gests. 

The difficulties which grow out of the more 
obvious and flagrant abuses of the question- 
naire by untrained and inexperienced people 
are not insuperable. Inexperienced, even vicious 
people, sometimes misuse the automobile, but 
no general outcry is raised against this method 
of transportation. In this, as in the question- 
naire problem, cooperation in regulation is the 
solution. A careful search of available liter- 
ature criticizing the use of questionnaires re- 
veals nothing, either in the form of theoretical 
argument or of objective evidence, which justi- 
fies a sweeping condemnation and complete 
abolition of the questionnaire as a tool of edu- 
cational investigation. 

Let us next consider the case for the ques- 
tionnaire. The case against this method of in- 
quiry, particularly when confined to its use by 
competent investigators is not a serious one. 
But what can be said in favor of the question- 
naire? 


The Case For The Questionnaire 
“By their fruits ye shall know them” has 


many applications. Has the questionnaire 
yielded significant fruits in the form of con- 
tributions to educational knowledge and prac- 
tice ? 





1 Koos, Leonard V. The Questionnaire in Education. Macmillan Co., 


* Ibid. page 57-68. 
* Kelley, oy ge L. Scientific Method, page 43. 


rge S. The Social Composition of Boards of Education. Supple mentary Education Monographs, 


nts, 
Chicago, Chicago, Ill., July, 1927. 100 pages. 


Koo’s recent book* presents important evi- 
dence on this His 
581 representative, published researches in 
Of these 581 studies, 
one-fourth, 


question. study analyses 
the field of education. 
143 or nearly 
principally on questionnaire data. Only two of 
the 143 were found to have “ 
that the question blank has 


depend wholly or 


little or no value.” 
Koos concludes 
proved to be a source of significant educa- 
tional data, and that many valuable studies in- 
cluding some of exceptional merit have made 
use of this method. 

Kelley * the 


from a different angle. He asked a small group 


recently approached matter 
of college scientists in a variety of fields the 
following question: 

In your opinion is a questionnaire ever an appro- 
priate instrument for use in the conduct of serious 
research in any scientific field? 

From seventy-five to a hundred percent of 
each of eight scientific groups catechized gave 
at least a qualified yes to the foregoing ques- 
tion.* Kelley closes thus: 

Let me conclude this investigation of the value of 
the questionnaire by saying that unless and until 
experimental science relieves us of the need of 
human judgments, or removes from our minds an 
interest in unique events, this wayward child of 
science, feeble as it is, will remain an indispen- 
sable helper. It will thus be always needed, and we 
can but hope that it will curb its intrusive dis- 
position and mend its unseemly ways.’ 

Illustrative questionnaire investigations. A 
moment’s thought will convince those in touch 
with educational investigation, that many sig- 
nificant studies lean heavily upon data obtained 
by questionnaire. 

Counts has published several investigations 
of recognized merit largely based on question- 
naire results. His study of boards of educa- 
tion ° is a recent example. After making gen- 
erous allowance for a certain degree of ad- 
mitted inaccuracy of the data, one accepts this 
investigation as reliable. It presents the most 
complete analysis of the make-up of school 
boards made to date and raises a series of fun- 
damental questions concerning their composi- 
tion. It would have been impractical to have 
undertaken this study except by questionnaires. 


1928, p. 138. 


No. 33, Univ. of 


[11] 








Another study based largely on question- 
naire data is that by Buckingham concerning 
teacher supply and demand." In spite of short- 
comings which the author points out in this 
study, it deserves to rank as a major contribu- 
tion in an important field, and one about which 
the teaching profession will hear much during 
the next decade. 

Terman’s studies of genius” represent an 
entirely different field in which considerable 
use was made of questionnaires. Whatever in- 
terpretation one may make of the facts pre- 
sented in this study, and upon whatever side 
one may array himself in the merry battle of 
nature versus nurture, he can hardly fail to 
place this among the more important studies of 
recent years—modifying or over-turning as it 
does many popular fallacies concerning that im- 
portant element of our human resources, the 
gifted child. 

Koos is another well-known investigator who 
has used the questionnaire extensively. A series 
of question blanks constituted an important 
means of assembling the data upon which he 
based his authoritative study of the junior col- 
lege. Few would question the great value of 
Koos’ investigation of this important move- 
ment. Surveys such as this would either be pro- 
hibitively expensive, or much less comprehen- 
sive, but for the questionnaire. 

Briggs’ early and widely influential book on 
the junior high school * used questionnaire re- 
turns from many schools as one of several 
methods of securing data. 

Charters and Waples used the questionnaire 
as one of several methods of securing data in 
their teacher training study.® This fundamental 
investigation presents a picture of the eval- 
uated activities and traits of certain types of 
teachers and should be of large value in the 
construction of professional courses for teach- 
ers. 

Questionnaires were one of the principal 
tools of ot in the recent modern foreign 


1 Buckingham “| 
Ohio State 5 otues 
? Terman, Lewis M. et al. Genet 
Stanford Uetvertty Press, Stanford 
See: Koos, Leonard V. The Junior C 
Series Number 5. Minneapolis, 





us, Ohio, 19 1926. 
yy Fa 1926. 
1924, ‘erp 


a 





of this in 


language study. Volume four ° 
gation, dealing with enrollment in foreigy |p. 
guages, is largely based on questionnaire re. 
plies. Few will doubt that this inquiry js oj 
major significance to secondary schoo! an 
lege development in the United States. 

Judd’s study of the length of eleme: 
education’ based partly on data collected 
through question blanks arrives at this impo 
tant conclusion: 


Ample evidence is present in the experie: 
the United States and Canada that elementary 
cation of satisfactory degree of richness of « 
can be provided in seven grades. 


The Land Grant College survey now being 
conducted by Arthur J. Klein of the Unite 
States Office of Education secured much of its 
data from a series of carefully prepared, but 
none the less long and detailed questionnaires 
When this study becomes available shortly, it 
will doubtless constitute the most fundamental 
survey of an important section of higher edu- 
cation which has yet been made. 

The foregoing illustrations which might be 
duplicated many times * are sufficient to prove 
that questionnaire investigation does result in 
contributions of major significance. 

The questionnaire as a trail-blazer—\ be 
foregoing paragraphs present evidence as to 
the value of the questionnaire as employed in 
fundamental educational investigation usual|; 
by the college specialist. It is also used in an- 
other form of investigation which, though not 
as fundamental as careful research, is never- 
theless important. 

The question blank is frequently of par- 
ticular value as a trail-blazer to educational 
investigation. Suppose one is contemplating « 
study, let us say, of the progress which has 
been made in developing differentiated curric- 
ulums to meet the varying aptitudes of second- 
ary school attendants, and his method is to be 
an intensive case study of the content and 


y and Demeed ie Teacher Training.” Bureau of Educational Research Monograph, No. 4. The 
of caine, Vel ee and Physical Traits of a Thousand Gifted Children.” 
ols. I and. Il, Rend Publications of the University of Minnesota, Education 


eacher Training Study. The University of Chicago Press, Chicago, 1929 


. Secondary Schools and Colleges of the United State 


Se ae 'y Education, Charles H. Judd, Chairman, Supplementary Educationa 


Also see: Koos, Leonard V. Junior ‘College M nement. Ginn and Company, a. 1925. 436 p. 
‘ Briggs, Thom as H. The ~q to A School. Houghton, Mifflin Co. Boston, 1920. 350 p. 
5 Charters, W. W ree Commonwealth T. 
> nd Others. Enrollment in the Foreign 
Modern tay g ge Study. by oA The Macmillan Co. 1928, 
= graphs No. 3 34, 5 ee of ¢ Fem aft hn “Chicago, 1927. 167 
“7.3 not referred to, which 


. Strayer, 


have cout some use of the questionnaire, are those of the Edu 


), the Studies in Adult Education (Edward L. Thorndike 


tonal tonal Fina ce nguiry Comm Director’ 
cmillan; a the studies of the Commonwealth Fund, see page 512 of School and Society, April 28, 1928 
[12] 





Kaa 


ewes te re | 








RNG 5 ON RR EGC AT NEL 


| 





the 
wh 
gat 
fect 
whi 
‘ 
nalr 

the 
ence 
tine 
ten¢ 
prol 
like 
ques 
obta 
supe 
thei 
T 
educ 
catic 
arisi 
boar 
whic 
Wh: 
shou 
ion, 
naire 
secul 
It 
logic 
antic 
tical 
pract 
be o 
devel 
of ta 
conti: 
01 
there 
that | 
for tl 
a neg 
questi 
could 
gation 
1K 


ected 


Nn 
1por- 


mntent 


being 
nited 
of its 
, but 
aires 
ly, it 
ental 
edu- 


ht be 
prove 
ilt in 


—The 
as to 
ed in 
sually 
n an- 
h not 
lever- 


par- 
tional 
ing a 
h has 
urric- 
cond- 
to be 
t and 
4. The 
ildren.” 


lucation 


10, 1929 
d States. 


cational 
/ Educa- 


ike and 
1928. 


affect of differentiated curriculums in ten high 
schools. It would be sheer folly to select ten 
high schools at random for this purpose. 
Rather, a brief exploratory questionnaire should 
go out to a large number of high schools in 
order to discover those which have done most 
to develop differentiated curriculums. Then 
the study should go forward in the ten schools 
where conditions promise most for the investi- 
gation. The questionnaire is frequently an ef- 
fective method of locating the veins of ore 
which may be worked with most profit. 

The school superintendent and question- 
naires—Another part of this bulletin considers 
the problem which superintendents of schools 
encounter in dealing with questionnaires. Per- 
tinent here, however, is the fact that superin- 
tendents and their staffs are among the most 
prolific distributors of questionnaires. It is 
likely that they will continue to use them. The 
questionnaire is frequently the only means of 
obtaining certain types of information which 
superintendents of schools and members of 
their staff must have. 

The school executive is the spearhead of 
educational progress so far as practical appli- 
cation is concerned. New issues are constantly 
arising, and he must frequently advise his 
board on short notice concerning a policy about 
which little or nothing is in printed form. 
What is he to do? It is stupid to say that he 
should base his advice solely on his own opin- 
ion, when he may circulate a brief question- 
naire among a few fellow superintendents and 
secure other judgments on the policy at issue. 

It is unlikely that scientific research, or 
logical reasoning—either or both—will ever 
anticipate all of the problems which the prac- 
tical school executive will encounter. Common 
practice and experience will doubtless always 
be one of the guiding forces in educational 
development. The questionnaire is one means 
of tapping this experience. It will, and should 
continue to be employed for this purpose. 

Questionnaires often the only way—Finally, 
there remains the very important consideration 
that there is frequently no practical substitute 
for the questionnaire. Koos concluded that only 
a negligible proportion of the 143 fundamental 
questionnaire researches which he analysed 
could have used any other method of investi- 
gation.’ The same may be said concerning most 

1 Koos, Leonard V. The Questionnaire in Education. page 64. 





of the carefully prepared questionnaires of the 
trail-blazing type—those which nominate de- 
sirable subjects for more careful investigation. 


Likewise, questionnaires circulated by school 
executives, those which collect information as 
to current practice and secure judgments on 
educational innovations, frequently seek un- 
available information which it is not practical 
to secure otherwise. 

Summary and conclusion—The first part of 
this chapter views the objections raised against 
the use of questionnaires in educational investi- 
gations. It has been stated that this method of 
inquiry requires much effort by the investigator 
for inconclusive results, assembles data that 
are unreliable and inaccurate, places too heavy 
a burden on respondents, and is generally poorly 
applied. Analysis of the foregoing objections 
indicates that they consist mainly of criticisms 
of the abuse rather than the right use of the 
questionnaire, and that there is insufficient basis 
either in the form of theoretical criticisms or 
objective evidence to justify a sweeping con- 
demnation of this instrument of educational 
investigation. 

The case in favor of the questionnaire is 
also examined. Evidence presented indicates 
that questionnaire studies have contributed sub 
stantially to the store of educational knowl- 
edge. The questionnaire serves a useful pur- 
pose in discovering veins of ore which deserve 
intensive working by research specialists. It 
is also considered an essential tool by many 
school executives in securing information con- 
cerning school practices and procedures. It 
would not be practical to obtain much of the 
information assembled by questionnaire which 
contributes to the foregoing significant out- 
comes through any other means. 

The abolition of this method of inquiry 
would work an unnecessary hardship on the 
scientific investigator in education as well as 
upon the practical school executive. No gen- 
eral abolition is necessary. The abuses which 
have risen out of the circulation of question- 
naires can be corrected by voluntary, coopera- 
tive regulation along lines suggested in a sub- 
sequent chapter. It is important that this 
method of correction be taken. Otherwise, one 
of the methods of inquiry which has substan- 
tially contributed to educational advance will 
be cut off at the source. 


[13] 





Better Preparation of Questionnaires 


Elsewhere in this Bulletin it is suggested 
that a principal cause of the disfavor in which 
many hold the questionnaire method of in- 
vestigation is the lack of skill with which it is 
used. This section will suggest certain guides 
for the preparation of questionnaires. These 
guides are based partly upon the experience of 
the Research Division in preparing question- 
naires for investigations sponsored by the Na- 
tional Education Association and partly upon 
the advice of previous writers on this subject.* 
Before listing specific suggestions, attention 
will be given to certain preliminary conditions 
which the investigator should settle before he 
begins to prepare his blank. 


Preliminary Considerations 


The topic investigated should be a worthy 
one. It is extremely difficult to set up objective 
criteria for judging the worth of a proposed 
subject of study. Yet it is true that some ques- 
tionnaires deal with educational questions 
which the great mass of competent critics 
would classify as significant, while others waste 
energy collecting data on matters of little im- 
portance or, as has been said, marshal battalions 
of data to prove something which everybody 


knew already and which makes little difference 


anyway. The advice of one or more experi- 
enced investigators is perhaps the best guide as 
to the worth of a subject proposed for ques- 
tionnaire study. If several experts agree that it 
is a topic which can best be studied by this 
method, then there is some warrant for pro- 
ceeding. 

The foregoing is particularly pertinent for 
the inexperienced investigator such as the typ- 
ical graduate student. It will be less helpful to 
the practical school executive confronted by 
the necessity, real or supposed, of quickly as- 
sembling data to shed light on some moot ques- 
tion of policy being considered by his school 
board. For example, it may appear a trivial 
matter to some people whether it is the practice 
to divide a teacher’s annual salary into nine, 
ten, twelve, or twenty-four payments. But the 
school administrator sometimes is unexpectedly 
confronted by a situation where he feels he 





1 See the bi phy, pages 47-9. 
2 See pages 44-7 for sources of information. 


must have facts and opinions on such a poin 
and in a hurry. What is he to do? Perh. 
the end circulate a questionnaire. But be{,, 
deciding to do this it would be well +. 
these questions: 

1. Would a specia! delivery letter o: 
gram to some recognized agency of educatio; 
information ? bring the material desired ? 

2. Is it not likely that the flare-up \ 
prompts the questionnaire will have passed 
fore the information needed can be assembled 

3. Will not a little hard thinking on thy 
subject by the superintendent and members of 
his staff be as safe a guide as the median of 4 
series of discordant practices, or the average of 
off-hand opinions, which are frequently what 
one gets in reply to a questionnaire on trivia! 
topics ? 

4. Will not a few hours’ search in the loca! 
school or public library be more productive 
than the same time spent in issuing and tabu- 
lating a hastily prepared questionnaire ? 

If the answer is no to each of the foregoing 
questions then perhaps it is justifiable to dis- 
patch a questionnaire for the information de- 
sired. But the burden of proof is on the in- 
vestigator who takes the time of busy people 
to fill out a blank assembling data on ephemera! 
issues of local interest. 

Adequate sponsorship should be secured— 
Discriminating respondents consider the spon- 
sorship carried by a questionnaire to be an im- 
portant criterion in deciding whether toanswe: 
it or not. If from a recognized agency, usuall; 
jealous of a good reputation, it is likely to have 
been carefully prepared. ‘These are among the 
offices whose sponsorship carries a measure 
of guarantee that the questionnaire is worth 
answering : 

1. Office of superintendent of schools par- 
ticularly when his staff includes a person qual- 
ified to conduct educational research. 

2. Schools or departments of education func- 
tioning through a research bureau or a pro- 
fessor who personally sponsors the inquiry. 

3. Governmental agencies concerned wit! 
education, state and national. 


[14] 





agenc 
out q 
Gr 
thesis 
or m 
profes 
oft a 
launc! 
Ag 
shoul 
questi 
of the 
partic’ 
investi 
Mo 
tionna 
tionna 
and ¢ 
blanks 
most o 
these } 
The 
has be 
edge i 
knowle 
a subje 
questio 
tions it 
become 
questio: 
of scho 
of no ¢ 
the 8-4 
making 
The 
the pro 
studied. 


A lit 


) such refere 


topics may 
re 


See 
1928. Vol. 


4. State and national education associations 
with facilities for expert guidance in research. 

The majority of questionnaires sponsored by 
agencies comparable to the illustrations above 
are carefully planned, and will directly or in- 
directly render some return to those who an- 
swer them. 

The questionnaires issued by 


most local 


teachers’ organizations are poor. Officers of 


these organizations would do well to secure 
the aid of some qualified representative of an 
agency such as those listed above before sending 


out questionnaires. 

Graduate students 
thesis material should secure the advice of one 
or more qualified investigators such as their 
professors or the assistance and endorsement 
of a recognized agency of research before 
launching a questionnaire. 

Agencies with facilities for sponsorship 
should take this responsibility seriously. The 
questionnaire should not only carry the name 
of the sponsoring organization, but that of a 
particular representative, qualified to appraise 


of education seeking 


investigations. 

Most unnecessary and poorly prepared ques- 
tionnaires can be kept from circulation if ques- 
tionnaire-makers will seek expert advice on, 
and official endorsement of, their inquiry 
blanks, and if respondents return unanswered 
most of the questionnaires which do not carry 
these passports. 

The investigator should know the field—lIt 
has been said that he who takes little know]l- 
edge into a strange land carries back little 
knowledge. Likewise, he who knows little about 
a subject will be able to gain little through a 
questionnaire. In order to ask intelligent ques- 
tions in a field of study it is necessary first to 
become a student of the field. To construct a 
questionnaire inquiring about the organization 
of school systems, which assumes the existence 
of no other type of school organization than 
the 8-4-4 plan, would be folly. Yet inquiries 
making such assumptions have been circulated. 

The literature and research bearing upon 
the proposed investigation should be carefully 
studied. One is repeatedly surprised at the 


amount of material which is already available 
on nearly every educational question. It may 
not be the last word, and may merely testify to 
the need for more study on the same topic, but 
it deserves consideration. If it does not receive 
it, the principal outcome of the questionnaire 
may be to advertise the ignorance of the in- 
vestigator to those who know the field. Before 
drafting a questionnaire, therefore, prepare as 
complete a bibliography * on the topic as is pos- 
sible, and evaluate the references cited therein. 

The scope of the problem should be carefully 
delimited—The student of tre- 
quently has the experience of beginning to 


education 


study a subject which at first appears to be a 
unit question, only to find that it soon divides 
and ramifies 
which must be dealt with one at a time. This 


into a whole series of issues, 
fact justifies the advice: clearly define the prob- 
lem and scope of your study before you draft 
the questionnaire. Study the topic being in- 
vestigated until its various strands are sepa- 
rated, and a deliberate choice is made of those 
which are to be the subject of inquiry. 

Observance of this principle usually calls 
for a clear-cut decision as to whether the ques- 
tionnaire is to have wide scope and eschew de 
tails, or cover a limited field and delve deeply. 
Below are some of the questions of a successful 
questionnaire * which sought to touch lightly, 
but to range widely over the field of teacher 
personnel : 


Married Women as Teachers 

Has your board of education a rule or policy 
against the appointment of new teachers who are 
married women? 

MN sx ce en sae I 

(Indicate answer by check mark.) 

Does your school board require women teachers 
who marry to give up their positions at once? 

rere reer At the end of the 
ee eer / 
(Indicate answer by check mark. 


school year? 


Written Examinations 


Does your school system conduct written examin- 
ations in any form in connection with employing 
new teachers? 


1A librarian will be able to cite the guides to educational literature most helpful in preparing a bibliography. Also consult 
such references as numbers 18 and 24 in the bibliography in this Bulletin. Reasonably recent and complete bibliographies on many 
topics may be obtained from sources of information given on pages 44-7. : 
1928. Wore 251, “Practices Affecting Teacher Personnel,"’ Research Bulletin of the National Education Association, September, 
. Vol. VI, 


No, 4. 


[15] 





(Indicate answer by check mark.) 
In connection with promoting employees within 
the service? Yes... 
(Indicate answer by check mark.) 


Sick Leave 


What part of the regular salary is paid to teach- 
ers taking sick leave? (Check below.) 

Full salary 

Half salary 

Full salary, less pay of substitute 

Some other fraction of salary 


If sick leave with some salary is allowed: 
a. How many days are allowed in_ one 


b. Is the allowance cumulative, i. e., may the 
days not used during one year be added to 
those of later years? 


(Indicate answer by check mark.) 
c. During how many years may sick leave allow- 
ance accumulate? 
(Indicate no. years.) 


The-questionnaire method is at its best when 
it employs questions such as the foregoing. It is 
less satisfactory in securing details of operation. 
Personal interview or individual correspond- 
ence is often the only reliable method of getting 
such information. It is hard to go wrong in an- 
swering the preceding questions. But when one 
attempts to develop one of these into a series of 
detailed questions, he should anticipate a re- 
duction in the reliability of replies. 

The wide-ranging as opposed to the deep- 
delving questionnaire is particularly valuable 
in uncovering pay-ore territory for future in- 
tensive investigation. For example, the ques- 
tions above on examinations would reveal the 
cities worth investigating if a study were pro- 
posed as to the use of written examinations in 
selecting and promoting teachers. Several 
studies in the teacher-personnel field have made 
their take-off from data revealed by the ques- 
tionnaire partly reproduced above. 

Whether one’s inquiry is to deal intensively 





or extensively with a field will depend on +h. 
object and scope of the study. If this has beep 
clearly thought out, one will know when ; 
use the extensive as opposed to the intensiy. 
type of question. 

Data should not be collected by quest 
naire if otherwise obtainable—lf the invest; 
gator will take thought on this point before 
preparing a questionnaire form, he may dis 
cover other means for getting the materia! he 
needs. It may be found that intensive, persona! 
study of a few situations will produce more 
valuable results than a questionnaire sent 
broadcast. School records and reports, govern 
mental reports, and the publications of educa- 
tional foundations and associations contain 4 
wealth of raw data of value to the investigator, 

One who has carefully studied published ma- 
terial in the field of his inquiry is not likely to 
ask for information already available. ‘he {ol- 
lowing statement is pertinent with regard to 
the unnecessary use of questionnaires: 


In the first place let us remember that the ques- 
tionnaire is often a lazy man’s method of getting a! 
leged information that could be much better ob- 
tained in some other way. It is easy to sit down in 
the quiet of one’s study and make out questions fo: 
someone else to answer. But such a method is ver) 
wasteful of the time and energy of the one to 
whom it is sent and is frequently wasteful of the 
time of the maker, for these questionnaires not in- 
frequently find their way into the waste-paper 
basket, and many of them should. One has no right 
to take the time of someone else in answering ques 
tions when he himself can find the answer else- 
where. Nor, on the whole, can he consistently ask 
someone to spend his valuable time in work that 
will not presumably prove of value to him or his 
school. After he has decided upon the data he 
wishes, he should search through all available ma- 
terial to see whether these data cannot be obtained 
through printed reports. If they can and he does 
not have the reports, he should send for them and 
get the data for himself. If the data given in the 
reports are not in the exact form in which he wishes 
them, he should change his own plan to fit the form 
used, where this is possible.” 


The follow-up should be anticipated—l\t \s 
usually desirable to send at least one follow-up 


1 The following is a good cxamole of a study which, Ay ey eer on but a small number of cases, exceeds the value of mon) 


Teachers questionnaire studies: Rufi, John. ““The Smal! H 


Contributions to Education, No. 236, Bureau of Public: at 


eachers College, Columbia University, New York City, 1926. 145 p. 
Commi 


ones, Arthur J., Chairman, National 
with Suggestions for 


U. S. Government 


ttee on Research in Secondary Education, Am Outline of Methods of Resea’ 
figh School Principals and Teachers. U. S. Bureau of Education, Department of the Interior, Bulletin, 1° 
Printing Office, Washington, D. C. 1927. 31 p. 


[ 16] 





impo 
hood 
full c 
such ; 


Te 
turn 
the u: 
conte! 
sponse 
vestig 
study 
the fo 
his col 

Lin 
follow 
begin 
questic 
waitin 
in the 

The 
low-ur 
of the 
the de 

Plan 
early 
idea of 
The au 


| followe 


that no 
lack of 


ported 


standsti 
pleting 
avoided 


to ten | 


 suggeste 


count of; 


! Koos 
3 Toop 
* Linds 


letter a short time after a questionnaire has 
been mailed. The vigor and number of follow- 
ups will depend upon the questionnaire. If fail- 
ure to respond is an outgrowth of some factor 
of selection, then a follow-up campaign is nec- 
essary. But the danger of a selected response 
does not always hold, as Koos points out: 


This danger by no means always obtains. It is 
important for the investigator to consider the likeli- 
hood and influence of selection and, if a practically 
full count seems essential, take the necessary steps, 
such as repeated inquiry, to secure it.’ 


Toops’* study, in which a 100 per cent re- 
turn on a questionnaire was obtained through 
the use of six follow-up letters, supports Koos’ 
contention that the danger of a selective re- 
sponse is not present in all questionnaire in- 
vestigations. The essential conclusion of Toops’ 
study is that additional replies brought in by 
the follow-up letters did not essentially change 
his conclusions. 

Lindsay's * investigation suggests that when 
follow-up letters are to be used they should 
begin to go out within two weeks after the 
questionnaire has been mailed, rather than 
waiting longer, and until there is a slackening 
in the receipt of replies. 

The cost of providing and mailing the fol- 
low-up letters should be anticipated as a part 
of the total cost of the investigation and before 
the decision to make the study has been made. 

Plans for tabulating should be formulated 
early *—-Too often investigators have little 
idea of the time required to tabulate replies. 
The authors of a number of the questionnaires 
followed up by the Research Division reported 
that no use was made of the replies because of 
lack of facilities for tabulating. Others re- 
ported the work partly completed but at a 
standstill due to the lack of facilities for com- 
pleting the job. Such an embarrassment can be 
avoided if the investigator will have from five 
to ten questionnaires filled out in the tryout 
suggested on page 22, and will keep an accurate 
count on the time required to tabulate them. 


' Koos, Leonard V., The Questionnaire in Education, p. 167. 


An estimate can then be made of the total time 
required to handle the expected number of 
replies. If it is more than one is equipped to 
furnish with reasonable promptness, then the 
questionnaire should be shortened and the aum- 
ber sent out reduced. 

The method of disseminating results should 
be settled—This involves typing, and duplica- 
tion in some form, printed or otherwise. The 
cost of duplicating the results of an investiga- 
tion in quantity sufficient so that there can be 
a reasonable dissemination of results will be 
considerable, particularly if it involves tabular 
material. ‘he probable number of tables, their 
size and character, and the amount of running 
text should be estimated ahead, and an esti- 
mate of the price for mimeographing or print- 
ing obtained. 

Finally, there is the problem of mailing the 
study, or at least of notifying respondents of 
its completion and availability. This should be 
an integral part of every questionnaire investi- 
gation. At the very least, the questionnaire 
should contain a question such as “Do you de- 
sire a copy of the tabulated replies to this 
questionnaire ?” 
tively should be taken care of. Preferably, the 


and all who respond affirma- 


questionnaire should make a definite promise 
that a copy of the results will be mailed to all 
respondents, or at least that respondents will 
be promptly notified when results are available. 
The latter practice is acceptable when the study 
appears in a magazine or as a separate publica- 
tion, the cost of which is too great to permit 
free distribution. 

When the material is to be used by a gradu- 
ate student in a thesis it is enough, and usually 
preferable, that a brief summary of the results 
be mailed to the respondents. But enough copies 
of the complete study should be available so 
that a copy may be borrowed by an interested 
respondent. Few things have done more to 
bring the questionnaire method into ill repute 
than the practice of some universities ° of per- 
mitting graduate students to bury a few type- 
written copies of the results of questionnaire 


. Toops, H. A. “Returns from Follow-up Letters to Questionnaires.” Journal of Applied Psychology. 10: 92-101; March, 1926, 
* Lindsay, E. E. ‘‘Questionnaires and Follow-up Letters.” Pedagogical Seminary 28: 303-7; September, 1921. 


* Also see pages 21, 25, and 36. 


5 The Research Division in seeking to secure copies of unpublished graduate dissertations for the use of important committees 


of the National Education Association has, in some instances, found 
from some college libraries. The question is raised: If a study is of sufficient vakie to justi 


it a time-consuming ey to borrow these investigations 
y the granting of an advanced degree 


should it not be a requirement that it be printed in full, if for the doctorate, and at least in summary form if for the masters degree? 


[17] 








| 
| 


studies, some of which are of real significance, 
in the university library. 

If the investigator is convinced that the in- 
formation he wants will shed light on a worth- 
while question, that his questionnaire is well 
sponsored, that he has a sufficient mastery of 
the subject to prepare an intelligent question- 
naire, that he has carefully delimited the scope 
of his investigation and therefore knows just 
what he wants, that he can get the desired data 
only through a questionnaire, that he is in a 
position to handle the replies with reasonable 
promptness, and to render respondents a proper 
return for the time which they will donate, 
then only is there justification for actually 
drafting the blanks. 


Preparing and Sending the Question- 
naire 


The following guides should be suggestive. 
They are not invariable principles but can be 
observed with profit in most cases. 

The questionnaire should preferably deal 
with matters of fact—This method is often at 
its worst when it attempts to discover truth by 
collecting subjective opinions or judgments. 
It is extremely difficult to anticipate all the 
implications and qualifications which will affect 
the respondent as he seeks to reply to a subjec- 
tive question. The following may appear to be 
a simple question: 

Do you believe school systems should adopt the 


platoon school type of organization? 
BO eentakied Fan daares iy Aa aS aa iyo 04 Sin ° 


But when one comes to answer it a swarm of 
considerations arise to befuddle the respond- 
ent. Just what is meant by a platoon school ? 
Is this form of organization equally effective in 
all grades? Would it not be better to experi- 
ment first in a few schools? Are not the school 
plants of many cities such that it would be im- 
possible to install the platoon plan, except at 
its worst? At about this time the respondent 
either flips a coin to determine his answer, or 
throws the questionnaire into the waste-basket, 
The prejudice felt against the opinion-seeking 








questionnaire is stated by one school exe 
as follows: 


The true questionnaire devotee never asks 
information desk at the North Station, “Wh. 
the next train leave Boston for Springfield 
seeks the opinion of all experienced travelers . 
in the seats of the waiting room and, with his « 
rule busily working, he hurries to the train oat, 


The following are some of the reasons 
suggest the desirability of reducing the nun 
of questions calling for subjective replies ; 
minimum : 


1. Questionnaires seeking opinions get a relative) 
poor percent of reply.’ 

2. The hastily expressed opinions of a large nun 
ber of judges of varying merit are probably of 
less value than the mature judgment of one o; 
two experts who have the question clearly j; 
mind, and who are able to qualify their answer 
appropriately. 

3. Answers to questions of opinion if thoughtfully 
given are time consuming, and cannot be dek 
gated to a clerk. 


The foregoing should not be interpreted as 
a blanket condemnation of a certain valuable 
type of questionnaire investigation in which 
the attempt is made to study opinions as such 
without regard to their innate value. In such 
cases opinions. become a species of fact worthy 
of study in and for themselves because of the 
light which they may throw on problems. || |us- 
trations of studies of this type are cited be- 
low.® 

Both Koos* and Kelley defend the ques- 
tionnaire as an instrument for determining 
consensus of opinion. After pointing out th 
difficulty of securing exact measures of human 
opinion, Kelley says: 


“The instrument for attacking this problem is the 
questionnaire. Is this not a superb anticlimax 
Here I have described the most difficult field of re- 
search, and that possibly the most pregnant for 
human welfare, and propose for its handling th 
weakest device that has ever obtruded its prying 
and at times discourteous self into the respectabl 
field of science. Though I must admit al! of this, 


I would ask how in the realm of mental values is 
any approach conceivable except that which ascer- 


- ——— E. W. “The Plenary Inspiration of the Dotted Lite." Educational Review 69: 1-4; January, 1925. 


Koges even on page 36. 


mg Teachers as to Right and Wrong.” Teachers College Record 31: 24-36; October. 192° 


fliam Hall. het Citizens Know About Their Schools. Contributions to Education, No. 279. Bureau of Publication: 
Teachers ers College, oa University, New York City. 1927. 86 p. 


he Questionnaire in Education, page 147. 


[18] 


PP Cy 


a NA OA AND ey ble scenes a 





+ ener 


Oe Natasa nents acs icin amen eee oe PL RRR AT T=" om — ap 


ex] 
tiol 
tha 
imy 


ana 
ple 
nes: 
que 
the 
hav 
opir 
resp 
shot 
C 
ask 
of t 
ably 
be t 
of | 
sear 
whic 
form 
the : 
C: 
tionn 
answ 
medi 
wise 
quest 
numl 
prince 
subje 
cials, 
consi 
plies 
and t! 
this r 








able 
hich 
such 
such 
rthy 

the 
llus- 

be- 


jues- 
ning 
> the 
man 


is the 
max? 
yf re- 
t for 
g the 
rying 
ctable 
this, 
ues is 
ascer- 


tains what the valuing agents assign to be the 


values?” * 

[experts can perhaps use the questionnaire 
effectively in securing a valuable consensus of 
human opinion on certain topics. ‘he begin- 
ner in the field should reduce the number of 
questions asking for subjective opinion to a 
minimum. 

Especial care should be used in selecting a 


group of respondents who are competent to 


express an opinion on the subject of investiga- 
tion. The number of replies is less significant 
than who replies—quality is of far greater 
importance than quantity. 

Ask only for data which respondents can 
and will give—Koos emphasizes this princi- 
ple under the criteria of ability and willing- 
ness.2 When dealing with matters of fact, the 
questions should ask for information which 
the respondent may reasonably be expected to 
have available. When dealing with matters of 
opinion or judgment, the competence of the 
respondents to give the desired judgments 
should be considered. 

Questions dealing with matters of fact may 
ask for information in the personal knowledge 
of the respondent, or for information reason- 
ably available from school records. Care should 
be taken to keep to a minimum the number 
of questions which will require extensive 
searching of school records. The extent to 
which an official will go in searching out in- 
formation not readily available will depend on 
the apparent value of the investigation. 

Care should be taken to include in the ques- 
tionnaire blank only questions which can be 
answered by the person addressed or by his im- 
mediate staff. For this reason it is usually un- 
wise to include in the same blank a series of 
questions which must be circulated among a 
number of people, for example, the high school 
principal, the general supervisor, the special 
subject supervisor, or other specific school offi- 
cials. The answering of such an inquiry entails 
considerable correspondence to round up re- 
plies from the several sources of information 
and therefore places a burden on someone. For 
this reason, and because the percent of replies 





! Kelley, Truman Lee. Scientific Method. p. 39. 
* Koos, op. cit., p. 99. 
* Koos, op. cit., p. 99. 


is likely to be low, questionnaires which re- 
quire replies from several different persons 
should be eschewed whenever possible. When 
they are necessary, the cooperation of the super- 
intendent of schools should be enlisted in the 
project before the questionnaire is sent to a 
city. 

Questions are sometimes asked which it is 
dificult for anyone to answer except by a wild 
guess. The following illustrations are from 
actual questionnaires: 

When Cumulative Sick Leave has been well 
established, do you consider the annual amount for 
sickness allowed by the Board would be materially 
greater than before the time of making Sick Bene- 
fits Cumulative? 

If you have in use the single salary plan, what 
was the effect of its introduction on the salaries of 
high school teachers? 


Is your janitor interested in the welfare of the 
children? Is the manner of his voice pleasing? 


The inclusion of such questions shakes the 
confidence of the respondents in the competence 
of the investigator. Replies to such questions 
are of little value. 

Then there is the question which indiscrimi- 
nately asks ages, or other personal questions 
which many dislike to give. Sometimes an hon- 
est reply might later embarrass the respondent, 
for example: 

Does the school board ignore the superintendent 
in many instances and thus assume authority right], 
belonging to the superintendent, yet holding the 


superintendent responsible for the general success 
of the schools? 


The willingness of the respondent to reply) 
needs to be anticipated. Questions which cal] 
for information of a highly personal nature, 
which call for answers reflecting on others, 
and which prematurely commit one on highly 
controversial issues are likely to reduce the 
percent of reply. When it is necessary to in- 
clude such questions a promise to report re- 
sults in such form that they can not be traced 
to individuals, or a waiver of signature, ma\ 
increase the willingness of the respondent to 
give complete and accurate replies.® 

The questionnaire should be as short as pos- 


stbhle—Short questionnaires tend to obtain a 








higher percent of return than long ones. ‘The 
present investigation included twenty ques- 
tionnaires containing five items or less, which 
received a median return of 78.5 percent, as 
compared to a 69.0 percent return on the 136 
investigated. Table 9, page 34 shows that, in 
general, the percent of return decreases as the 
length increases. 

An effort should be made to eliminate every 
question not essential to the problem. Many 
questionnaires contain one or two queries the 
replies to which can not be utilized. The time 
of everyone concerned would have been saved 
by the omission of such questions. 

On the other hand, the few replies to a long 
and detailed questionnaire may be more valu- 
able to the investigator than a nearly complete 
response to a shorter questionnaire on the same 
topic. The investigator should weigh the pos- 
sible advantages of a high percent of replies to 
a short questionnaire, as compared with a large 
amount of information from the few generous 
respondents who will answer a long one. 

In certain cases the long questionnaire, which 
has been carefully prepared, which is adequately 
sponsored, and which deals with a subject of 
general interest, receives a high percent of re- 
turn. For example, the longest and probably 
the most difficult questionnaire in the 267 ob- 
tained for this study received a return of 88 
percent. The blank related to the cost of in- 
structional supplies, and their distribution to 
the various grades in the school system. It was 
16 pages long, and called for 1,833 ‘items of 
financial statistics. It had been preceded by a 
letter to superintendents of schools, describing 
briefly the plans for the study, and inviting 
them to cooperate. The superintendents of 125 
cities expressed a willingness to cooperate, and 
110 of them furnished the information re- 
quested, 

Questions should be simply and clearly 
worded—No respondent should be expected to 
puzzle out a questionnaire. One should be able 
to make his wants known in simple language. 
If he cannot give a brief and clear explanation 
of the plan of his inquiry, it is probably not 
suited to the questionnaire method, and he 
should either simplify his plan or change his 
method of collecting data. 








The simplest form of inquiry, and the | 
most easily answered, consists of a series 
complete questions, each of which can st 
alone, independent of context. For the s| 
questionnaire, this is the most desirable { 

When an inquiry is very long and detai 
howeyer, it may be simplified by casting it i; 

a tabular form with directions to the respon 
ents as to the method of answering. In | 
paring a tabular form for recording statistic! 
data, great care is required in the wording 
column headings and definition of items i: 
cluded. The blanks used by the U.S. Offic. 
Education * for the collection of statistics on 
school personnel and finances illustrate th 
working out of this principle in a practica| 
situation. 

The tabular form may also be used to sim 
plify the long questionnaire dealing with non 
statistical data. For example, a questionnaire 
dealing with functions in health education may 
list a series of possible activities carried on in 
this work, preceded by a request that the re 
spondent check those functions performed in 
his local school system. Or it may be somewhat 
more detailed, providing separate columns 
headed “classroom teacher,” “school nurse, 
and “school doctor,” with directions to check 
the functions performed by each. 

But the investigator must guard against 
too great complexity. To carry further the 
example of the questionnaire on health func- 
tions, he might ask the respondent to use one 
check when the function is performed fre- 
quently, and two checks when performed in- 
frequently, and to indicate in another column 
by letters “A,” “B,” and “C,” the relative im 
portance of having the function performed at 
all. Unrestrained, he may even add additional 
columns to be filled. in, headed “Grades In- 
cluded,” “Number of Pupils Reached,” “Yea: 
When Function Was First Performed,’ and 
“Desirable Outcomes of the Function,” the 
latter column provided with a numbered code 
of possible answers, and instructions to ente! 
the code number instead of the actual answer. 

Questionnaires of this complicated type fre- 
quently ask for data of value, but they deteat 
their own purpose through their bewildering 
plans for getting responses. The investigato: 


' See page 289 ff. of “School Records and Reports,”’ Research Bulletin of the National Education Asscciation, Vol. V, No. 5. 


[20] 








ORE, ve, 


ee = a at i Es 


a 





ide 
res 
fac 
inf 
oul 


in 





cannot expect his own interest to be equalled 
by the respondent to the extent of puzzling out 
an involved and intricate scheme for recording 
facts. 

The necessity for clear wording applies not 
only to the general plan of the questionnaire, 
but to the separate questions within it. Each 
question should be so clearly worded that all 
respondents will interpret it the same way. It 
is only by the most painstaking efforts, pre- 
liminary trials and revisions, that questions can 
be prepared which are entirely clear, and which 
will mean the same to all respondents. Such an 
ideal is hard to achieve, but the value of the 
results of the study will depend largely on this 


factor. 

That the form in which a question is asked 
influences the reply received has been pointed 
out by investigators dealing with the question 


in various situations. The order of a question 
in a series also affects the answers which will 
be given.? The significance of these facts should 
be understood by those who would use the 
questionnaire effectively. 

The questionnaire should require a minimum 
of writing—Every effort should be made to re- 
duce the writing necessary to answer the ques- 
tionnaire. It should be worded so that the an- 
swers may be expressed by yes or no, check 
marks, underlining or encircling, by figures, or 
by short answers of a few words. 

The investigator wants to be sure, however, 
that his questions are suited to this type of re- 
sponse. In wording a question to be answered 
yes or no, one should see that it can be an- 
swered with an unqualified affirmative or nega- 
tive. The following are examples of yes or no 
questions to which few school officials could 
give a categorical answer: 


Is there more than one clerical assistant in each 
school? 

Is the cost of building foundations below street 
level, underground pipes, excavations, and side- 
walks, included in the total amount of insurance 
carried? 

Is your department so organized that you con- 
sider the interpretation of the course of study to 





your staff members more important than the con- 
struction of the curriculum? 

If you have kindergartens, do they accommodate 
ill eligible pupils? 

When listing a series of answers for check- 
ing, underlining, or encircling, care should be 
taken to cover all the possible cases, or to pro- 
vide space for “Other Response.” In many cases, 
it will be desirable to ask a direct question to be 
answered with a few words. When such a short- 
answer question is so clear that its meaning 
is the same to all respondents, the replies are 
comparable. It is more likely to bring to light 
unusual practices than questions which list a 
series of alternative answers. 

Care must be taken, however, to limit the 
scope of the short answer question so that its 
replies can be tabulated and summarized. The 
question which invites the respondent to write 
an essay produces replies which cannot be 
tabulated. Rather than essay questions, it is 
better to ask short answer questions, and to 
provide a space for “Remarks” where addi- 
tional details will be recorded by those willing 
to give them. 

Responses should lend themselves to tabula- 
tion—Rugg has well summed up the advice 
which is pertinent on this point: 

The answers to “general” questions seldom can 
be tabulated and definitely interpreted. It is a safe 
rule to follow that data which do not lend them- 
selves to tabulation and statistical treatment are of 
negligible value to the investigator. That is, an- 
swers should be definite and susceptible of tabular 
classification and this should be a controlling cri- 
terion in the planning of questions.° 


Emphasizing the thought still further Rugg 
suggests that the investigator work out the 
tabular forms for entering the replies to his 
questionnaire at the same time that he prepares 
his blank: 


This means the planning of the specific headings 
of the tables to be compiled, and will require defi- 
nite decisions concerning the arrangement of ques- 
tions and the probable types of returns to secure. 
Preparation of the tables will lead to a clearcut, 
logical arrangement of questions, so put together 
as to facilitate a clear presentation and discussion 


1 Weidemann, Charles Conrad. How to Construct the True-false Examination. Contributions to Education, No. 225. Bureau of 
Publications, Teachers College, Columbia University, New York City. 1926. p. 41-67. 
Muscio, Bernard. “The Influence of the Form of a Question.” British Journal of Psychology 8: 351-89; September, 1916. 


? Mathews, C. O. 
Psychology 128: 34; February, 1929. 


The Effect of the Order of Printed Response Words on an Interest Questionnaire."’ Journal of Educational 


* Rugg, Harold. Statistical Methods Applied to Education. Houghton, Mifflin Company, Boston, 1917. p. 47. 


[21] 








in the report. A little time spent at this stage of the 
work will aid much in the later organization of the 
completed discussion.’ 


Each question should be so worded that it 
can be answered by a single statement. Ques- 
tions are frequently asked which require two 
answers, such as the following: 

Are you contemplating a change in history text- 
books, if so which one are you considering? 

Do you study occupations as a group or are the 
boys and girls receiving separate instruction? 

Has any legal action directly applying to pri- 
mary grades been taken and what? 

Frequently the respondent will read only 
half of such a double question and answer yes 
or no to the despair of the tabulator. 


A careful examination of individual ques- 
tions with this principle in mind will often re- 
veal some of the replies which it will be im- 
possible to tabulate, or which will consume 
much of the tabulator’s time and give uncer- 
tain results at best. Such questions had best be 
eliminated or recast. 

The questionnaire should be given a prelim- 
inary tryout—The extent of the preliminary 
trial will depend on conditions. It should be 
remembered, however, that the dependability 
of the answers may be strengthened materially 
by thoroughly trying out the form. One easy 
test, which should be made of even the simplest 
questionnaire, is to try it out locally. Before 
duplicating the form, the investigator should 
have typed copies of the questionnaire answered 
for his own local school system. If a few copies 
are sent out to nearby cities, the replies will 
almost inevitably suggest needed modifications 
of the form. Questionnaires goiny to other 
groups may similarly be tried out first on a 
small sampling of those to be circularized. 

The National Committee on Research in 
Secondary Education suggests that before 
sending out a questionnaire it should go 
through the following stages: 

(a) Very careful formulation by the author and 

arrangement in the form to be used. 

(b) Submission to some expert for advice and 

correction. 


(c) Tryouts on teachers or others not primarily 
concerned—disinterested persons. 





Ibid, p. 





(d) A tryout of the revised questionnaire 
group as nearly like the ones to whom it is 
be sent as possible. These tryouts will! ofte; 
show the inaccuracies of statement, th, 
equivocal questions, and other undesirab 
features that can be corrected before ¢\y, 
questionnaire is actually sent out for repli, 


If all questionnaires were to pass throu 
the foregoing stages, their authors would not 
only save their own time and that of their re- 
spondents, but would get far more valuable 
products for their labor. 

A questionnaire should meet certain stana 
ards of mechanical form—A questionnaire 
which is poorly arranged and carelessly du 
plicated is likely to raise doubts in the minds 
of the recipients as to the ability of the in 
vestigator to follow a study through to a suc 
cessful conclusion. The response will therefore 
be reduced. A neat, well-arranged question 
naire encourages an accurate, businesslike reply. 
These are among some of the standards of 
mechanical form which should be observed: 

a. The questionnaire should be neatly re- 
produced. Typing, if well done, is acceptable if 
only a small number is being circularized. 
Mimeographing, multigraphing, and electric 
typewriting are duplicating processes which 
produce copies resembling typewriting, and are 
satisfactory for short questionnaires going to a 
limited number. Printing is the most desir- 
able method of reproducing questionnaires, and 
tends to get a better reply. It is especially ad- 
vantageous when the questionnaire is long, in- 
volves tabular and columnar arrangements, and 
is going to a hundred people or more. 

b. The form should contain spaces in which 
to enter the name, offical position, and full ad 
dress of the respondent. Allow ample space. It 
is poor policy to irritate the respondent at the 
outset by allowing only half enough room for 
his name and address. Even with such space 
provided some will forget to use it. It is de- 
sirable to know who has filled out a question- 
naire in case later correspondence is necessary 
and for other reasons. If envelopes are pinned 
to questionnaires as they are opened, the post- 
mark will aid in identifying them if signatures 
are omitted. Earlier in this section is discussed 


1 50. 
2 Jones, Arthur J., Chairman of National Committee on Research in Secondary Education. An Ouéline of Methods of Resea’ 


with 
24. p. 2 = 


igh School Principals and Teachers. U. S. Department of Interior, Bureau of Education. Bulletin, 1926, \ 


[ 22] 











tama ssi 








the circumstances under which it may be de- 
sirable to waive the signature and other iden- 


tification. 

c. The questionnaire itself should contain 
the name and address of the investigator. 
Otherwise, the transmitting envelope is likely 
to be thrown away, the letter of transmittal 
lost, and the questionnaire left homeless. Some- 
times they are sent to the wrong address and 
cannot be forwarded as there is nothing to in- 
dicate who sent them out. 

d. Ample space should be left for the an- 
swering of each question. A crowded question- 
naire allows no space for the answer longer 
than average, and encourages incomplete and 
slovenly replies. Experience has shown that a 
two-page questionnaire, generously spaced for 
replies, gets a better reception and response 
than if it is crowded into a single page. The 
tryout of the form will suggest the amount of 
space necessary for the answer to each ques- 
tion. It is desirable to have a line or two after 
certain short-answer questions, and at the close 
of the questionnaire for “remarks.” Unusual 
practices, significant qualifications and other 
material often of great importance to the in- 
vestigator, will be written in if space is pro- 
vided. 

The purpose of the investigation should be 
stated—More than one-third of the question- 
naires studied failed to state the intended use 
of the results. Recipients are likely to be sus- 
picious of investigations which have no visible 
purpose. It is not enough merely to be “inter- 
ested in the problem of sabbatical leave,”’ or to 
state “I am making a study of child accounting 
systems.” Rather state ‘‘we are in need of tliese 
facts to help us establish a sound policy in the 
payment of teachers during sickness,” or “I 
shall use this in a dissertation at Blank Univer- 
sity, which is being written under the super- 
vision of Professor Doe. A summary of the re- 
sults will be mailed promptly to all respond- 
ents.” Frankness on this point gives assurance 
that the material requested is to serve a useful 
purpose and begets cooperation. 

The statement of purpose may be given in a 
letter of transmittal or in an introductory 
paragraph on the questionnaire. It should be 
brief, but adequate, and should not be omitted. 





Buckingham would go even farther, and his 
advice is particularly pertinent if the question- 
naire is long and involves the assembling of 
material not immediately available. He states: 

The receiver of the material should be taken 
fully into the investigator’s confidence and should 
form his own judgment of the value of the under- 
taking. It is generally wise to write in advance to 
the intended recipients of the questionnaire and to 
acquaint them with the project. Ask them if they 
believe it to be worth while and if they will fill 
out the necessary blanks. Otherwise what right have 
we to “fire” a battery of questions at a group of 
busy people? To do so is both inartistic and dis- 
courteous. There is a brusque sort of effrontery in 
printing all the hard questions one can frame on a 
given subject and thrusting them without so much 
as “by your leave,” before the face of a distant 
and therefore defenseless victim. The stamped self- 
addressed envelope adds the last excruciating touch. 

Investigators could raise considerably the esti- 
mation in which the blanks are held and the will- 
ingness of people to make them out if they would 
add to their technical skill in constructing forms 
an element of restraint and a modicum of courtesy.' 

Other desirable practices. There are certain 
other customs which are often desirable under 
the principle of courtesy, if for no other reason. 
They follow: 

a. A separate letter of transmittal is usually 
desirable. It offers an opportunity to explain 
the purpose and extent of the investigation and 
to give other pertinent information. It should 
be brief and to the point. Such a letter may be 
addressed in general terms, for example, “To 
the Superintendent of Schools” or “To the 
Teacher of Mathematics.” Still better, the 
name and address of the individual receiving 
the questionnaire may be typewritten on a mul- 
tigraphed form. Individual letters to each re- 
spondent are desirable in some cases, partic- 
ularly when the number addressed is less than 
fifty and the cost is therefore little more than a 
form letter. If any large number of question- 
naires is being sent out the cost will be least 
for a general form letter, and most for per- 
sonal individual letters. In some cases, par- 
ticularly when the questionnaire is very short, 
the letter of transmittal may be a part of the 
questionnaire itself, and the cost thereby re- 
duced. 

b. It is usually desirable to send two copies 
of the inquiry form to each respondent. The in- 


' Buckingham, B. R. “The Questionnaire.” Journal of Educational Research 14: 54-8; June, 1926. 


[23] 








formation requested is frequently of value to 
the respondent. The extra copy allows him to 
record it in proper form, and to have an exact 
record of his report in case any question later 
arises. ‘his courtesy is desired and sometimes 
demanded, particularly by offices of superin- 
tendents of schools. The cost of the extra copy 
is not great when the form is mimeographed or 
printed. On the first page of the blank should 
appear the note: “Two copies of this blank are 
sent, one to be returned to the address indi- 
cated, and one retained for your files.” 

Extra copies will also be needed if a follow- 
up is to be conducted in order to increase the 
percent of reply. In order to provide duplicate 
copies for respondents, to conduct the follow- 
up, and for other reasons which sometimes de- 
velop, it is usually desirable to print three times 
as many blanks as the number of persons to be 
circularized. The cost of securing an ample 
number is less if it is done at first, rather than 
reduplicating or reprinting the blank. 

c. It is usually desirable to enclose a return 
envelope. The size of the envelope should be 
determined by the size of the question blank. 
The envelope should usually be stamped. There 
are cases when this may not be necessary, par- 
ticularly if the questionnaire is going to well 
equipped offices, and the results of the study 
will be of obvious value to the respondent. 

Summary and conclusion—This_ chapter 
draws upon the experience of previous investi- 
gators and of the Research Division and sug- 
gests certain practical guides to questionnaire 
investigation. It is stated that previous to pre- 
paring a questionnaire one should be convinced 
that the information sought will shed light 





upon a worthwhile rather than a trivial iss, 
that the investigation is well sponsored, that | 
has a sufficient mastery of the field to be sty 
ied, that he has carefully delimited the scope 
his investigation, that the desired data can 
obtained only through questionnaires and th; 
he has facilities with which to handle the rep} ic: 
with reasonable promptness and to rende: 
spondents a proper return for the time the) 
will donate. Satisfied as to these essential pre 
liminaries the investigator may proceed to ¢} 
drafting of his blank observing these criteria 
of good questionnaire preparation: The ques 
tion blank should preferably deal with matters 
of fact, which respondents are able and willing 
to give, rather than with opinion or judgment. 
The number of questions should be reduced to 
a minimum by eliminating non-essential in 
quiries, be clearly and simply worded, and re- 
quire a minimum of writing by the respondent. 
The response to questions should lend them 
selves to tabulation. This and other desirable 
features can be best guaranteed bya prelim 
inary tryout of the blank. The questionnaire 
should meet certain standards of mechanical 
form such as legibility, neatness, proper spaces 
for responses, should be fully identified, and 
should be accompanied by a brief and frank 
statement of the purpose of the investigation. 
It is desirable also to provide a letter of trans- 
mittal, extra copies of the blank, and a return 
envelope. These criteria of good questionnaire 
preparation should not be looked upon as in- 
variable principles but usually can be observed 
with profit particularly by inexperienced in- 
vestigators. 


[ 24] 








—— 


arene 


: 
P 
; 
: 
t 


th 
wl 
sul 
sta 
toc 
cif 
im, 
nal 
to 

wil 


vic 


Sot 
Re 
gra 
anc 
sho 
qui 
nar 
gat 
shot 
the 
inte 
regi 
tent 
den 
be ¢ 
he | 
the 
vest 

7 
No 
one 
a qu 
will 
tion 
such 





Reporting Questionnaire Results 


Many books and articles are in print which 
describe proper statistical and other procedures 
in handling educational data.t One who would 
make effective use of facts assembled by ques- 
tionnaire should be familiar with these writ- 
ings. They apply quite as much to the handling 
of data from questionnaires, as to those from 
other methods of investigation. 

This chapter will not attempt to recapitulate 
the general principles which should guide one 
when he reaches the stage of presenting the re- 
sults of an investigation. Rather, at the risk of 
stating the obvious and of dealing with details 
too trivial to mention, it will make certain spe- 
cific suggestions, the observance of which would 
improve many studies based upon a question- 
naire. These suggestions should be of most help 
to the relatively inexperienced investigator 
with meager resources for securing expert ad- 
vice in reporting the results of his study. 

The report should be properly identified— 
Some of the questionnaire reports reaching the 
Research Division, particularly those in mimeo- 
graphed form, contain little to indicate where 
and by whom they were prepared. Every study 
should have a title, stating the subject of in- 
quiry as definitely as brevity will permit. The 
name, official position, address of the investi- 
gator, and the date of the investigation also 
should be given. These are essential even though 
the belief is that the report is purely of local 
interest. It is best not to be too modest in this 
regard. The investigation may come to the at- 
tention of outside people, even if only by acci- 
dent, and may be of interest to them. It will 
be of more value to the outside investigator if 
he has some means of identifying the origin, 
the age of the data, and the position of the in- 
vestigator. 

There should be an organized presentation. 
No rule of thumb can be laid down as to how 
one should proceed in presenting the results of 
a questionnaire investigation. Different studies 
will deserve varying treatments. An organiza- 
tion of one’s report under a series of headings 
such as the following will usually be desirable: 


a. Statement of purpose or problem of study. 





A description of the unsolved question on which 
the study sheds light. 

b. A brief description of how the data were 
collected, including an estimate of the relia- 
bility of the information secured, under such 
topics as: the number of questionnaires sent 
out, how those circularized were selected, the 
percent of reply, an estimate of the influence 
of the factor of selection if less than 100 per- 
cent replied, and anything else which an honest 
evaluation of the data should suggest. 

c. Presentation of data, including an ex- 
planation of what each table deals with, and 
what it seems to show. 

d. Statement of the conclusions based upon 
the data. This should not be confused with the 
preceding point. For example, it is one thing 
to state the fact that 60.7 percent of the cities 
in the United States do not appoint married 
women to new teaching positions, and it would 
be quite another thing to conclude from this 
fact that most superintendents of schools 
believe married women are poorer teachers than 
unmarried. In the first instance we have a fact 
which cannot be challenged, the latter is a con- 
clusion based on the fact, which may or may 
not be justified—probably the latter. 

e. Summary and conclusions. This should 
round up the study in a few succinct state- 
ments. ‘he ease with which this takes shape is 
an excellent test of one’s clarity of thinking and 
writing up to this point. A well organized 
study is easily summarized, even by a stranger 
to the study. If it is hard for the investigator 
to tell the conclusion of the study, how can a 
reader expect to discover it? 


The foregoing headings should not be slav- 
ishly followed in writing up questionnaire re- 
ports. Variations and modifications will fre- 
quently be desirable. In a long investigation 
the foregoing headings may virtually reoccur 
over and over again as one table after another 
is presented, evaluated, and interpreted. Just 
how one organizes a study is less important 
than that it should be organized. 

Inexperienced investigators sometimes pre- 
sent data solely in a series of tabulations. This 
is inadequate even though the tables are well 


1 See such references as Nos. 1, 10, 14, 23, 24 and 30 in the bibliography. 
[ 25] 











set up, and have clear and complete titles, exact 
though brief column headings, and a short cue 
as to how to read the table. More than this is 
needed. ‘The investigator presumably has done 
much thinking on the subject of the inquiry, 
and should be in the best position to discuss 
the significance of his data. He should briefly 
tell what he thinks each table shows, partic- 
ularly if it seems to indicate a significant trend 
or practice. Any shortcomings of his figures 
should be frankly stated, and possible miscon- 
ceptions by those who may make casual in- 
spections should be guarded against. Such in- 
terpretation of each table is desirable even 
though very briefly done. 

Tables should be properly totaled and 
checked. The various columns of each table 
should be accurately added and balanced and 
appropriate medians or averages entered. When 
frequency distributions ? are involved, the total 
of the vertical and horizontal sub-totals should 
balance. ‘These points may seem obvious, but an 
inspection of a random sampling of tables will 
show that some fail to observe them. 

The totals of columns, themedians, averages, 
or other measures of central tendency are the 
raw material out of which the investigator 
composes the text of his discussion and inter- 
pretation of tables. 

The master tabulation should be included. 
It is usually desirable to present the data from 
a questionnaire in a series of short tables which 
bring out the tendencies and trends on one 
point at a time, but which obscuré deviations 
from general practice, and make it impossible 
to identify the data of any specific table in re- 
lation to particular cities or respondents. This 
practice is satisfactory, but something in addi- 
tion is usually desirable, namely, the detailed 
presentation in a master tabulation of the re- 
plies to each item of the questionnaire by indi- 
vidual respondents. Such a tabulation should 
list in the first column the individual respond- 
ents—cities, schools, or individuals—in some 
logical order, which will make for ease in 
locating the data from a given place or person. 
The succeeding columns should be allotted to 





Research in of the National Education Association, V 


the preparation of questionnaires, 


2 + bese Sift of reference No. 24 in the Bibliography for a pr 





the various questions of the inquiry, the col, 
heading repeating as fully as practical 
wording of the question concerned, The r 
of each respondent to each question can then 
indicated. Abbreviations of responses, ‘‘y’’ 
yes, “n” for no, and other space saving de\ 
should be used. If the questionnaire is long j; 
will be necessary to divide the master tabu |, 
tion into several sections. 

The ease with which a master tabulation 
takes shape is a good test of one’s success in 
preparing the questionnaire. If the questions 
have been properly prepared,® it will be pos 
sible to indicate answers in brief form.‘ If th, 
questionnaire was logically organized in the 
beginning—questions on the same topics being 
grouped together, and sub-questions following 
principal questions—the arrangement of the 
master tabulation will automatically follow 
the questionnaire arrangement. This wil! make 
for ease in tabulating and checking, and wil! 
afford logical dividing points if the question 
naire is so long that the master tabulation must 
be split into more than one section. 

The master tabulation will be of interest to 
but a small percent of those who consult th 
study and may, therefore, appropriately be sub 
merged at the end of the report. Its inclusion, 
however, may be justified on several bases: (| 
It has to be made anyway, if the questionnaire 
is to be properly tabulated, so its inclusion in 
volves only the cost of reproduction, (2) 
allows other investigators to identify unusual 
or pioneering practices and to investigate them 
more fully, (3) It makes for care in tabulat- 
ing, since the basis for the various tables in 
the main body of the report is found in the 
master tabulation, and if they do not check with 
it, errors may be discovered and corrected. Al! 
respondents are accounted for on each ques- 
tion—whether they reply or not. The reason 
why some respondents did not reply to certain 
questions can be hypothecated. (4) It makes 
for care in reporting. A respondent will hesi 
tate to make careless or inaccurate reports on 
the items of a questionnaire if it is likely his 


1 For an illustration of tables which are relatively as ey in a see “Practices Affecting Teacher Personn 


19 
on a “reauency distributions. 
ly pages 15 and 2 


section on particularly 
4 Essential qualifications of answers may be handled in brief footnotes at the foot of he master tabulation. 
[ 26] 

















[umn 
the 


V ices 
ng it 


bula- 


ation 
SS in 
stions 
‘ pos- 
f the 
1 the 
being 
wing 
f the 
ollow 
make 
will 
stion- 
must 


est to 
it the 
> sub- 
usion, 
: (1) 
naire 
yn. in- 
2) It 
usual 
them 
bulat- 
les in 
n the 
: with 
J. All 
ques- 
‘eason 
ertain 
makes 
_ hesi- 
rts on 
ly his 


sin will return to plague him later in the 
master tabulation. 

Below is illustrated the master tabulation of 
the questions given on pages 15-6 of this bulletin. 

Summaries of the data illustrated in the ac- 
companying master tabulation may be found 
in Tables 4, 5, 11, 12, 13, 14, 15, and 16, of 
“Practices Affecting Teacher Personnel.” Re- 
search Bulletin of the National Education As- 
sociation, Vol. VI, No. 4, September, 1928. 

If for any reason it appears impossible to 
include the master tabulation as a part of the 
report, one should at least give a list of the re- 
spondents to the questionnaire and indicate 
that the complete tabulation is available for 
inspection by interested investigators. 

The questionnaire should be reproduced— 
There should be appended to each report, just 
before or just after the master table, a copy 
of the question blank used in assembling data. 
In evaluating questionnaire data it is fre- 


quently as important to know just how the 
questions were asked as to know the responses. 
When an investigator includes both his blank 
and the master tabulation, all cards are on the 
table and the readers of his report are in a 
position to accept or reevaluate the conclusions 
based on his data. 

Summary—This section offers certain prac- 
tical suggestions regarding the preparation of 
reports based on questionnaire data. Such re- 
ports should be titled and otherwise properly 
identified, the presentation and interpretation 
of data should be carefully organized, a master 
tabulation giving answers of individual re- 
spondents to each question should be included, 
and a copy of the completed questionnaire ap- 
pended. These characteristics of a good report 
although obvious are sometimes lacking. Their 
intelligent observance by investigators would 
improve many questionnaire studies. 


FIGURE III.—ILLUSTRATION OF A MASTER TABULATION 


Chew 3¢000te soo 900 we pofulstirn 





Prarvued 
eaters 
tingle wwermery 





Mull 2x - 


APVitnatcne 
Conducted im Aarnibew 


pe : ; yee tion 


| 


ech bare 
age atthe 
Gare allowed with. 








a 


x 


NS 





caro teeh 


O¢ 
Gd 
vt sap 44 


: | 1%, 
: \ 


One 


=) 


Aumte tL 
ba 






























































> 


— 





Individual superintendents of schools have 
written, and the Department of Superintend- 
ence of the National Education Association 
has resolved, in protest against the number of 
questionnaires which annually reach the offices 
of public school systems. This situation im- 
pelled the Research Division of the National 
Education Association to begin in 1927 an in- 
\estigation of the questionnaire problem par- 
ticularly as it concerns the offices of superin- 
tendents of schools. 


Evidence Is Assembled 


In order to secure definite evidence as to 
the size and character of the problem the co- 
operation of a number of state, city, and county 
superintendents of schools was enlisted. The 
school systems asked to cooperate in the study 
were selected to represent various geographic 
areas and school systems of different types. 

These offices agreed, immediately upon the 
receipt of a questionnaire, to mail the Research 
Division a postal-card form? giving the name 
and address of the sender of the inquiry, and 
its subject and date of issuance. Upon receipt 
of one of these notices, a personalized form 
letter was sent to the investigator, asking for 
a copy of his questionnaire and explaining the 
purpose of the request. A follow-up letter was 
sent if a copy of the questionnaire was not re- 
ceived in six weeks. This procedure was con- 
tinued without essential variation for the 
school years 1927-28 and 1928-29. 

At the close of 1927-28, an inquiry was dis- 
patched to each investigator who had furnished 
a copy of his question form, asking informa- 
tion as to the number sent out, the number of 
replies received, and inquiring concerning the 





circulation A F. Harman, then County Supt. of Schools, 
W. E. ‘Morgan, Chief, Division 
Supt. ui Schools, San Francisco, Calif.; C. E. Greene, Director of 


Supt. of Schools, Hutchinson, Ka 


Go t nsas; Clarence 
Supt. of Schools, Wel ned Mose Millard 


and Research, Montclair, J. Bickett, Supt. of Schools, Trenton, N 

ERECTA Np og Ag hep hcg nat 
s, . H 

Va.; and Thomas Ww. Gosling, then a S of Schools, Madison, Wis. 


Others not elsewhere acknowledged who 
Leonard V. Koos, Belmont Farley, and S. D. * ween 
2 The form used is reproduced on 


al f eqpapetotemens is due to the following school officials who Spopereted in this study by reporti 
Montgomery, Ala.; Fred M. 
of Research and ——y- | ane & 


* H. , 
= itesetion =} “conn: Fr County Supt. of Schools, Towson, Md.; 5. Monroe Grave 8 
School 


Cc. Lefer, Supt. of A. L. Hartman, Director of Reference 





The Questionnaire Problem From the Viewpoint of the Superin- 
tendent of Schools * 


tabulation and dissemination of results.’ Ne 

a year later, in March, 1929, another ingui 
was sent to all who had supplied copies of thei, 
questionnaires seeking information as to 
made of the data they had assembled. 


The Evidence and What It Shows 


The number of questionnaires—Table | 
lists the cooperating school systems and | 
number of questionnaires reported in each case. 
After eliminating duplicate cards for the same 
questionnaire reported by different cities a total 
of 376 different questionnaires were reported 
by 14 superintendents of schools in 1927-28. 
In 1928-29, 364 different questionnaires were 
reported by a similar number of cooperators. 

The average annual number of question- 
naires reported by the superintendents’ offices 
of the city school systems in Table 1 is 45. The 
number received by individual cities varies 
widely. Five report less than twenty in a yea 
and four report more than seventy-five, the 
largest report from one city being 137. 

Approximately 370 different questionnaires 
were reported by the 14 cooperating offices in 
each of the two years. This is less than might 
have been expected, considering statements fre 
quently made as to the “‘floods” of question 
naires which annually descend upon the schools. 
A report from 14 centers, even though ran 
domly distributed geographically, cannot be 
accepted as a complete showing for the whole 
country, particularly when nearly all of the 
cooperating offices are located in large or mid- 
dle sized cities. It is safe to say, however, that 
it obtained a substantial portion of the ques- 
tionnaires of wide distribution, as well as some 
of those of restricted and local circulation. 


questionnaires i: 
mys Oaklan 


Hunter, then Supt. 
M. — 


mat Education, Sacramento, Calif.; 
nk or W. Ballou, Supt. of Schools, Washington, D. C.; 


ue, Lincoln, Neb.; 
. J.; William H. fading Supt. of Schools, Mt. Verno: 
Schools, Lakewood, Ohio; E. M. Otis, Supt. 

Pa.; J. H. Saunders, Supt. of Schools, Newport News 


have offered helpful suggestions in the preparation of this Bulletin are: Carter G: 


+ The form used in collecting this - + is reproduced on page 50. 


4 Form is reproduced on page 51. 


[ 28] 


nver, Colo.; Roger M. Thompson, eee i Re- 





ee - 


, 





| 


plet 
the 
cit rT 
the 
favi 
plet 
ave 
reas 
that 
the 
tha 


chaj 
sam 








‘le | 
1 the 
| Case, 
same 
total 
orted 
7-28, 
were 
ators, 
stion- 
»ffices 
. The 
varies 
, year 
. the 


laires 
es In 
night 
s fre- 
stion- 
100ls, 
ran- 
it be 
vhole 
f the 
mid- 
that 
ques- 
some 
n. 


ires in 


The number of questionnaires reported by 
individual cities is of interest. One of the co- 
operating cities reported more than 100 in a 
vear. The average for all cities was 45, or 
about one a week. 

The question should be raised as to whether 
all questionnaires received by the cooperating 
superintendents’ offices were reported to the 
Research Division. It is known that a com- 
plete report was not received from some. On 
the other hand, it is believed that in some 
cities, in which most of the mail passes through 
the hands of one alert secretary, conditions were 
favorable for receiving an approximately com- 
plete report. The five largest reports for a year 
average 89 questionnaires per city. It seems 
reasonable to conclude from the data presented 
that few offices of superintendents of schools in 
the United States receive on the average more 
than two questionnaires a week. 

All of the subsequent conclusions of this 
chapter are dependent upon an analysis of a 
sampling of the questionnaires which reach 


TABLE 1—NUMBER OF QUESTION- 
NAIRES REPORTED BY COOPERAT- 
ING SCHOOL SYSTEMS 





Number of 
questionnaires 
reported in 


Cooperators 


1927-28 | 1928-29 





1 2 





State Departments 
California* 30 
_ Connecticut 41 
City School Systems 
| Oakland, California* 
San Francisco, California** 
Denver, Colorado** 
Gc cc cases cs. 
Hutchinson, Kansas 
Wellesley Hills, Massachusetts** 
Lincoln, Nebraska** 
Montclair, New Jersey 
Trenton, New Jersey 
Mount Vernon, New York 


Lakewood, Ohio** 

Willoughby, Ohio* 

Philadelphia, Pennsylvania**.. . . 
Newport News, Virginia 
Madison, Wisconsin* 


County School S$ m 
|  Montgome Tenney, Alabama’*. 


Baltimore County, Maryland*.. 
| Miscellaneous 


Total number reported 


Total number, excluding du- 
plicates 

















*Reported in 1927-28 only. 
*Reported in 1928-29 only. 














the offices of superintendents of schools. It is 
believed that this sampling is sufficiently ran- 
dom to justify the conclusions which are drawn. 
In the cases where conclusions may have been 
significantly affected by non-random selection 
special notice of this possibility has been taken. 

Analysis of questionnaires received—A total 
of 376 different questionnaires were reported 
by the 14 centers cooperating in 1927-28. It 
was possible to secure copies of 306 of these. 
The 70 remaining questionnaire circulators 
failed to respond to two separate requests ask- 
ing for copies. Of the 306 received, 39 were 
eliminated for one of these reasons: (1) an in- 
complete copy was sent, (2) the inquiry was 
merely a general request for such material as 
was available on a given topic, and not a true 
questionnaire and (3) the request was of the 
nature of a regular periodic report required by 
the state department of education. Similar 
treatment of the 364 questionnaires reported 
for 1928-29 reduced the number to 265. There 
were thus available 267 and 265 bona-fide ques- 
tionnaires for 1927-28 and 1928-29 respec- 
tively, or a total of 532, certain features of 
which could be studied in detail. 

Although 740 inquiries were reported by the 
cooperators in the two years, only 532 could 
be studied in detail. To what extent did this 
reduction involve the factor of unfair selec- 
tion? The reduction was due to two causes, 
first, the elimination of certain inquiry forms 
which were not bona-fide questionnaires. Non- 
random selection did not enter here. Second, a 
small percent of the originators did not respond 
to requests for a copy of their blank. It is likely 
that the failure to respond was due in some 
cases to the fact that the blanks had been 
poorly prepared. Some selection probably en- 
tered because of this and other factors. Copies 
of approximately eighty percent of the inquiries 
reported were obtained. Non-random selection, 
therefore, could not have been present except 
to a minor degree. 

Who sends questionnaires?—Table 2 sum- 
marizes the data as to where questionnaires 
originate. Of 532 questionnaires which reached 
the offices of certain superintendents of schools 
over a two-year period, 27 percent were sent 
out by a city superintendent of schools or by 
members of his central office staff, i. e¢., assist- 
ant superintendent, director, or supervisor. An 


[29] 








additional 23 percent were sent out by other 
employees of school systems, i. e., principals, 
officers of teachers’ organizations, or individ- 
uals. Therefore, half the questionnaires were 
sent out by superintendents or members of their 
executive or instructional staffs. 

Approximately 23 percent were sent from 
the colleges by professors and graduate students. 
Private education associations and foundations 
originated 10 percent. The remainder, about 
17 percent, originated from a variety of agen- 
cies. 

Half the job of regulating the output of 
questionnaires which reach the offices of super- 
intendents of schools, therefore, must be per- 
formed by the superintendent and members of 
his executive or instructional staffs. Superin- 
tendents and their staffs, college professors, 
and representatives of a few educational asso- 
ciations and foundations by entering into some 
informal, cooperative plan, such as that sug- 
gested on page 38, can control the situation as 
it affects more than 80 percent of the inquiries 
which reach the offices of school executives. 

The field of the questionnaire—In order to 






for considerably more than half of those s 
ied. That this is not merely a reflection of 
fact that administrative problems account 
a large part of all educational research, js jp 
dicated by Koos’ data in Table 3, which show 
that 54 percent of printed questionnaire studi 
deal with administration, while but 31 percen 
of published investigations which do not em 
ploy the questionnaire method are in the field 
of school administration. .¢ 
It seems safe to conclude therefore, that the 
questionnaire is of especial importance to those © 
interested in educational administration, 
whether they be research workers or school | 
executives. The abolition of the questionnaire 


nal 
sho 


dec 





would close one of the important doors through TA 
which students of school administration, both | 
theoretical and practical, secure data essential J” 
to their purposes. r 


When are questionnaires sent out?—O 
tober to April seems to be the “open season.” 
None of the months during this period of the 
year seems to have a corner on the supply, [ 
according to the data of Table 4. In organiz- | |/— 
ing provisions for handling questionnaires in a [~ 


2 na RR tes 


_— 

































































discover the educational fields from which these superintendent’s office, it would seem wise to 
questionnaires sought information, they were set up facilities that wili be available through- | ( 
classified under a number of major educational out the school year. F 
heads. Table 3 summarizes the result, and How long are questionnaires?—The num P 
compares it with the findings of Koos who _ ber of separate questions asked, or items of in 
analysed 143 published questionnaire investi- formation requested, was taken as an indication ont 
i gations as to fields dealt with. Both investiga- of the length of questionnaires. According to _ 
' tions agree that administrative topics account this measure, questionnaires vary in length ~ | 
TABLE 2.—WHO SENDS QUESTIONNAIRES TO THE OFFICES OF SUPERINTEND- —— 
ENTS OF SCHOOLS? 
— — TAI 
1927-28 1928-29 ag i one 
j Originator of questionnaire 
t a ed Percent a =~ al Percent pf Percent 
1 2 3 4 5 6 as 
perintenden fii sins 
ee ee 66 24.7 78 29.4 144 27.1 Se 
62 23.2 60 22.7 122 22.9 Oc 
61 22.9 61 23.0 122 22.9 Ne 
25 9.4 30 11.3 55 10.3 De 
17 6.4 7 3.0 25 4.7 Ja 
10 3.7 9 3.4 19 316 Fe 
7 2.6 10 3.8 17 3 Si 
19 7.1 9 3.4 28 5.3 Ay 
——|———_ -- M. 
267 100.0 265 100.0 532 100.0 Bi ju 
' Read table thus: Of 267 different questionnaires received by offices of superintendents of schools in 1927-28, 66, or a 
24.7 percent were sent out by city superintendents of schools or members of central office staff; 62, or 23.2 percent were ; 
if sent out by principals, teachers, or other school employees, etc. Similarly read data for 1928-29 and for both yearscombined. | | 
1 Fi w 
hy — ee i * 











[30] | | 





from those which list less than five items to answer. Some very short and other very long 
those which include more than 500. Some are ones were well prepared and deserved careful 


nt aa : 
; half sheets, others are almost volumes. The answering. On the other hand there were 
isi P 7 . ; 
median number of items requested by the 267 atrocious examples of both long and short ques 
shows . ° . P ’ 
7 questionnaires of 1927-28 was 17.4. Over sixty tionnaires of the type which deserved to be 
studies 4 


percent had 20 items or less. Table 5 gives fur- assigned to the waste basket or returned un- 
ercen ; . . . . . : 
ther data on this point. Most questionnaires answered accompanied by an appropriate 









































ot em- , ; aa 
field seem to be reasonable in their demands insofar letter. 
e nel . ° . P ‘ 
as number of items requested is concerned. Form of the questions—It has been urged 
at of An inspection of the individual question- that time could be conserved through the use 
at the ; , ‘ ; 
tH naires did not suggest that brevity or length of questions which may be answered yes or no, 
> those . ° . ° ° e —_ , - 
rats should be adopted as an infallible criterion in or by the use of check marks. The 267 ques- 
ation, eee ; =U ‘ , ; ; 
hool deciding whether a questionnaire is worthy of _ tionnaires were examined to determine the form 
Scnoo 
mnaire 
\rouh [| TABLE 3—THE FIELDS OF EDUCATION IN WHICH QUESTIONNAIRES SEEK 
heh INFORMATION 
’ 
sential —— a ae 
Questionnaires to Published question- reer mg SS 
—Oc- superintendents in | naire investigations - tons ° oe wal 
1927-28 and 1928-29! | analysed by Koos? st a 
-ason.” Te y 4008 
ot the - || Number | Percent Percent | Number | Percent 
a, aman as : } tj Semeer_| Bere 
iy Fe » 
= + | St aa aR 74. 1.1 
es ina * Curriculum ‘ 7. 2 
" : . Methods of teaching...... ; 2'8 
Vise to . Supervision : 1. 2 
* . Tests and measurements. . 1. 2.3 
rough- . Psychology ‘ Kes ‘ | 7 
[ Ape yer | 5 
; | $ 
Research : | 5 
num- . NE vs hs kcina ces he ‘eR 7 
. of in- | 143 9 
ication | 
ling to Read table thus: Of 532 questionnaires sent to superintendents in 1927-28 and 1928-29, 398, or 74.8 percent sought 
- | information concerning administration; 91, or 17.1 percent sought information concerning the curriculum, etc. Simi- 


length larly read data for columns 4 and 5, and 6 and 7 
\ The Research Division has available detailed tables on which columns 2 and 3 are based. 
2 Koos, Leonard V. The Questionnaire in Education, Macmillan Company, New York, 1928, page 32. 














TEND- 


TABLE 4.—MONTH OF ISSUANCE OF QUESTIONNAIRES SENT TO OFFICES OF 
SUPERINTENDENTS OF SCHOOLS 





Issued in 1927-28 Issued in 1928-29 
Month issued ae SE eS ee visas ia 


) 
Percent Number | Percent 





w 


| & 


| 





| 


| WAM Am em AIS | 
| 


September 





MNOCWNNWOmm | 


| 


100.0 | § | 100. 


= 
— 

















Read table thus: Of 267 different questionnaires sent to offices of superintendents of schools in 1927-28, 8, or 3 percent 
| Were issued in September; 18, or 6.7 percent were issued in October, etc. Similarly read data for 1928-29 and for total. 
| 

















[ 31] 





le ea 








neat ae ae = oS 


ey Saas 






TABLE 5.—DISTRIBUTION OF _ 267 
QUESTIONNAIRES CIRCULATED IN 
1927-28 ACCORDING TO NUMBER OF 






































ITEMS OF INFORMATION RE- 
QUESTED 
Number of items yon og Percent 
requested naires of total 
1 2 3 
2-S.. 38 14.2 
eee 48 18.0 
Ps S whesvreweeh ed cnast 37 13.9 
16-20 38 14.2 
CL ct cg aed temo cares 16 6.0 
26-30. . 11 4.1 
31-40... 25 9.4 
41-S0.. 14 5.2 
51-60. 6 2.2 
61-80. . 8 3.0 
81-100 ‘smi waite are aca § 1.9 
101-200.... ere 9 3.4 
a a See 8 3.0 
More than 500 4 1.5 
ie 2.0% wtb Kad 267 100.0 
Median number of items... . 17.4 
Read table thus: Of 267 different questionnaires 
sent to superintendents in 1927-28, 38, or 14.2 percent 
requested information on from 2 to 5 items; 48, or 18 
percent requested information on from 6 to 10 items, 
etc. 














of question used, Four types of questions were 
recognized, according to the form of the re- 
sponse desired, as follows: (1) Yes-or-no ques- 
tions, those calling for a direct categorical an- 
swer, (2) check questions, giving respondents 
the opportunity to check or underline one of 
several answers or symbols of rank value given 
on the questionnaires, (3) short-answer ques- 
tions, requests for statistical information or 
short verbal responses, and requests that the 
respondent enter a number or a letter indicat- 
ing the rank or weight of various propositions ; 
and (4) essay questions, those calling for 
lengthy verbal responses of descriptive or ana- 
lytical nature, which could not be subjected to 
statistical treatment. 

It was found that 70.5 percent of the ques- 
tionnaires consist of combinations of the above 
types of questions, and that 29.5 percent use 
some one type of question exclusively. The vari- 
ous types appear with the following frequency: 
yes-or-no, and short-answer, 34.5 percent; 
short-answer only, 23.2 percent; yes-or-no, 
check, and short-answer, 14.2 percent; yes-or- 
no, short-answer and essay, 7.5 percent ; check 
questions only, 4.1 percent; check and short- 
answer, 4.1 percent; and other types and com- 
binations, 12.4 percent. Essay questions are in- 








cluded in 15.8 percent of the forms. Only 8? 
percent of the questionnaires are limited +, 
yes-or-no and check questions, but 64.9 p, 
cent use them together with other types , 
questions. 

The foregoing data indicate that approx 
mately 85 percent of the inquiries sent to < 
perintendents consist of questions which can 
be answered with a minimum of writing, as 
only 15.8 percent include questions which cal! 
for extended answers of the essay type. Al! of 
the short-answer questions were not as clear 
cut as they might have been, and some could 
be criticised from other points of view. [t | 
encouraging to note, however, that compara- 
tively few questionnaire senders are asking 
questions which call for extended answers of 
the type which first try the soul of the respond- 
ent and later defy efforts at tabulation. 

Character of information sought—\lt has 
been stated that these types of information 
may legitimately be collected by questionnaires: 
(1) Facts known to the person responding, 
(2) facts readily available from school records, 
and (3) judgment or opinion. Of the 267 
questionnaires of 1927-28, 51.5 percent asked 
for facts or figures only, 10.6 percent requested 
judgments and opinions, while 37.9 contained 
items asking for both. 

The chapter of this Bulletin dealing with 
the characteristics of good questionnaires pre- 
sents certain material bearing on the character 
of questions asked. Here it is sufficient to say 
that questions of fact should be limited to items 
which are ascertainable with a reasonable ex- 
penditure of time and effort. When questions 
do not meet this criterion they should be left 
blank. On the other hand, there are questions 
of fact which some school systems cannot an- 
swer which they should be able to answer. For 
example some systems have no record of their 
employed personnel on such essential items as 
training, experience and age. Such information 
automatically accumulates if new employees 
are required to fill out a personnel card. 

The difficulty with some questionnaire items 
therefore, is not that they are improper items, 
but that some school systems do not maintain 
the records which principles of sound manage- 
ment would justify. It is suggested, therefore, 
that when a carefully prepared questionnaire 
from a competent investigator requests items of 


1 See page 274 of ‘School Records and Reports,” Research Bulletin of the National Education Association, Vol. V, No. 5 
[ 32] 





: 





Re 


RRR  s 


| 


O 


cludi 


ion, . 
jecti' 
type 
fully 
respo 
tionn 
it ret 
spons 
cleric 
see kit 
as pi 
Ww hol 
in th 
ot go 
Di 
of du 
erally 
assert 
the f: 
numt 
tain | 
tionn 
sively 
numb 
in di 
classe 
answe 
word 
to be 
tionn: 
lem. | 
questi 
naire 
was t 
naires 
of bot 
Those 
reveal 
Cor 
tions 
when 
and 
differe 
The 


questi 





'Th 





factual information concerning a school sys- 
tem, which cannot be obtained except by a spe- 
cial investigation, it may be desirable to eval- 
uate one’s system of records.’ 

Of the 48.5 percent of questionnaires in- 
cluding some requests for judgment and opin- 
ion, a large proportion asked that certain sub- 
iective factors be ranked or weighted. This 
type of questionnaire, whereas effective if care- 
fully prepared, is often especially resented by 
respondents. To answer properly such a ques- 
tionnaire frequently not merely takes time but 
it requires the time of those in the more re- 
sponsible positions. It cannot be answered by 
clerical workers. This, as well as other opinion- 
seeking questionnaires, should be used as little 
as possible and then only by experts. This 
whole problem is discussed at greater length 
in the section dealing with the characteristics 
of good questionnaires. 

Duplication in questionnaires—The amount 
of duplication in questionnaire circulation gen- 
erally has been assumed to be large. It has been 
asserted that tremendous waste results from 
the fact that the same questions are asked by a 
number of different questionnaires. To ascer- 
tain the amount of duplication, the 267 ques- 
tionnaires collected in 1927-28 were inten- 
sively studied with particular reference to the 
number of times identical questions occurred 
in different questionnaires. Questions were 
classed as identical if they called for identical 
answers even though somewhat differently 
worded. The amount of duplication was found 


to be small. To be sure, 40 of the 267 ques- 


tionnaires dealt with phases of the salary prob-. 


lem. But the number of times the same salary 
question was asked by more than one question- 
naire was not great. The largest duplication 
was the request by seven different question- 
naires for the minimum and maximum salaries 
of both elementary and high school teachers. 
Those dealing with the curriculum similarly 
revealed few identical questions. 

Comparatively little duplication of ques- 
tions was found on any of the topics. Even 
when on the same topic, the approach varied 
and the particular information requested 
differed, 

The results of the tabulation of duplicating 
questions is shown in Table 6. It shows that 26 





questions were found which occurred on three 
or more questionnaires. Such was the duplica- 
tion discovered on 267 blanks, with a total of 
13,070 questions. 

This small amount of actual duplication 
does not mean that there is no waste in ques- 
tionnaires. If forty salary questionnaires give 
a poorer statistical picture of the salary situa- 
tion in the United States than the one com- 
prehensive study which the National Educa- 
tion Association makes biennially, then it is 
double waste to send out the forty, from forty 
different centers, with their limited means of 
composing, circulating, following up, tabulat- 
ing, and interpreting these blanks. From this 
fact it is not concluded that the circulation of 
questionnaires should be centralized in a few 
hands. The problem is one which calls for dis- 
crimination as to function. ‘To say that there 
are certain types of inquiries which can be most 
economically carried on by 
agencies and which ordinarily should be at- 
tempted only by such agencies, does not imply 
that there are no investigations which indi- 
viduals, and local centers can pursue with 
profit. Opportunity to circulate questionnaires 
should not be closed to the individual, but the 
individual investigator should use discrimina- 


well-equipped 


tion in selecting subjects to be studied by this 
method. 

Factors affecting percent of questionnaires 
answered—Information was sought from the 
267 questionnaire circulators of 1927-28 as to 
the number of questionnaires sent out and re- 
TABLE 6.—NUMBER OF DUPLICATING 

QUESTIONS IN 267 QUESTIONNAIRES 


ASKING A TOTAL OF 13,070* QUES- 
TIONS 





Number of questions 
appearing more than twice 


Frequency of 
occurrence 


2 








(Column 1, Table 5) by number of cases (Column 
2, Table 5) and adding. 

Read table thus: There were two questions each of 
which reoccurred in seven questionnaires; two ques- 
tions each of which reoccurred in six questionnaires, 
etc. 








| 
| * Calculated by multiplying mid-points of intervals 
| 





'The Research Bulletin cited in the previous footnote is a helpful source of information on records and reports. 


[33] 





SPOTS Ee 


as 


(os 


a * 


< 


SSS 








plies received. It was possible to secure a re- 
port from 136. The fact that only a little over 
half of the questionnaire-makers responded 
may be due to several factors, one of which is 
probably that some of those who had a poor 
reply did not wish so to report. The conclu- 
sions based on the 136 replies need to be in- 
terpreted with this in mind. The succeeding 
facts about the percent of reply received is 
probably for a selected group of the better 
questionnaires. 

Table 7, indicates that the typical question- 
naire of the 136 received a 69 percent reply. 
The reply varied somewhat according to the 
position held by the investigator. As might be 
expected, superintendents of schools are some- 
what more likely to answer inquiries origi- 
nated by other superintendents than by other 
persons. However, an examination of the de- 
tailed distributions of replies, not given here,’ 
shows that the percent of reply received de- 
pends more upon other factors than upon the 
originator. Some questionnaires sent by super- 
intendents received a reply of less than a forty 
percent, and others a 100 percent reply. The 
same may be said for those of college professors 
and other classes of originators. 

The month in which a questionnaire is is- 
sued also has some influence upon the reply 
received, according to Table 8. Generally 
speaking, it appears that the earlier in the 
school year a questionnaire is issued the more 


TABLE 7.—PERCENT OF REPLY RE- 
CEIVED BY VARIOUS CLASSES OF 














TABLE 8—PERCENT OF REPLY RE 
CEIVED AS RELATED TO MONTH 
OF ISSUANCE OF QUESTIONNAIRES 

















issua Number Median 
Month of mee issued percent 
of reply 
1 2 3 
September dpethessae-s 5 6.0 
ae 12 4 
November......... 22 73 
mber. 19 6.0 
eogeey Rieti aew abso « 20 59.0 
SS ae 15 69.8 
ES ee 16 66.0 
CaS y cick co Ses 6c 17 64.8 
SS ee ee 10 61.0 
RL a a 136 69.0 











Read table thus: The median percent of reply t 
5 different questionnaires issued in September w: 
76 percent; the median percent of reply to the 12 
different questionnaires issued in October was 74.3 
percent, etc. 

















likely it is to receive attention by a superin- 
tendent’s office. January seems to be the poorest 
time to cast a questionnaire upon the waters. 
Table 9 presents data as to the relation of 
the length of the questionnaire and the reply 
received. Generally speaking, the shorter the 
questionnaire, the better the reply. This rela- 
tionship is far from invariable, however, for 
the questionnaires asking for five items or less 
received a reply all the way from less than 20 
percent to 100 percent, while the longest of 
the 136, a veritable volume asking for over 
1800 items, received a reply of 88 percent. 
TABLE 9.—PERCENT OF REPLY RE- 


CEIVED AS RELATED TO LENGTH 
OF QUESTIONNAIRE 



































QUESTIONNAIRE CIRCULATORS 
Number of items of Number | Median 
tion requested of ques- | Percent 
Number | Median tionnaires | of reply 
Originator of questionnaire of ques- rcent — 
. demksives of reply 1 2 3 
1 2 3 5 or less. . 20 78.5 
EES in daleis ¢ Sch alee 16 71.0 
City superintendent or member Th ns SEs op aks « 22 71.0 
mPa PS < 544 acehew tne 42 75.4 Ses ee 17 68.5 
School principal or other school SE 15 66.0 
QIN, 5 kc os ceetiestcccis 23 66.0 Sh # ow'viee ev os 14 61.0 
College professors or students. . 29 60.4 ae ia lace are 4.0 6 71.0 
Private association or foundation. 19 58.0 RS re 10 76.0 
ee eae s 61.0 NG dK wh wiraveio.n 08 5 66.0 
CE Wels Ga eons Koes O6508 6 71.0 ere x $4.3 
Miscellaneous, ................ 9 78.5 501-1000. 2 ; 
1800-1900. .............. 1 
ee eee 136 69.0 — 
- he US 136 69.0 
Read table thus: The median percent of reply to i 
the 42 different questionnaires sent out by city super- Read table thus: The median percent of reply to 
intendents of schools or members of centrai office was 20 different questionnaires requesting information o: 
75.4 percent; median percent of reply to the 23 different 5 or less items was 78. J cay mw 3 the median percent of 
questionnaires sent by school — or other reply to the 16 requesting information on from 6 to 
school employees was percent. etc. 10 items was 71 percent, etc. 




















ee 


a 


a lg. 



































1 These tabulations are on file in the Research Division. 


[ 34] 



























































20 
of 


ay 


ver 


ey ee Berne, 


ela- 
for 
less 


AU 19430 AQ PoNss! SaureuUOT}senb 10; Bep pres Y} aloul 10} aiaM SolTeuUOT}sen Db 
} JO euros jo By[NSe1 1 SI yey ‘sesodind ¢¢ } 1 aIeM SaIIPUUOTISEND ZF Bey] 332 ‘suIa[qold [OOgIS 377s NO s710dal YIM UOTDeUUOD UI pesn aJem JUVOITEd ¢°6 JO “F ‘sta 
qos je uo $z10d 1m UOTIeU u arom 19d 4°99 10 ‘gz ‘eI [eIIWSD JO STequrem! 10 sjuapuUszUuLadNs AID Aq panss] SelteuUONSENd WUSIEYIP ZF JO *8NY? 3IQe? peoy 
asodind 9u q} 3 10} pasn 31am SaTTeEUTOT}SeNb smOs JO $7]NSal 9Y} BOUTS OO] JeAO ZSuryioMIOS Jie UUIN[O sq? UI syuadJed BYy2 JO STe}0} BY 
ILZt |O'TET| $77 |F'6 ot is’ 9 f 61 ee 6°71 7 rut 61 cst 9% |9 FT sz |8'8 St js't¢ 6s TRIO] 
| t | l l T 1 $Z f 1g £ Pst é Lil T L’l I Tez t st 
¢ ) 6 0°0 I 02 I 0°07 I 0°09 £ 0°09 ¢£ 
6 I ol ££e ¢ z 9°Ss s wiy Surysyqnd 
LI L 82 9'LI § Sit z s¢ 9 9'LI £ L°¥9 I | 9o'LT £ uonepunoy 
| } | | 10 UOoNRPosse uONPONp” 
zt i a Mat ae ¢ I €7 Ot j¢'tt | 9 061] 8 67 gl \8 b z €° Fl 9 TZ € Jossajo1d 10 JUapPNIs BBIaJO [J 
¢t €°9OIl) OF |O'L f £°7z I tI 9 9°TT s sé i 3 £°6 Li /£°6 + 2 7 zis 7 svaAoO|dula 19470 a 
} 10 Jayoee3 ‘yedpurd jooys9s ff LL 
zt TE! ss 611 iS 6 t $°6 F $°6 t 8 Ft z 1S 6 t ¢$°6 > L°99 87 Hes [e1ZUGD JO JoquisUl 
40 juspusazIuuedNs AID 
$2 £Z | wt 1Z OZ 61 gt 91 SI tI el a I or 6 | 8 l 9 Ss t £ z if 
1}u99 |} Jeq }u99 Joq | }u99 | Jeq | yue9) Jeq | }u9d| Jaq | yuad| Jeq | yu99) Jaq | yUe2| J9q yu92 | Jaq | jued | s9q | 3u0o| 30q | 
-18qg |-uiny |-19q |-winy |-J0g |-urn yy |-39q |-winy |-10g | -urny | -10g | -urny -139d -winy] | -J0qg | -mny ~19q | -umyy | -Jeg | -urny | Jaq | ~uiny 
soleu | | | } | | | 
-u0y | a, se = | — | a ee oe ee 
-sanb jo oie in | | sisoq} uonjeziues | weqoid we]qoid 10}BBYSSAU] JO SN}VIS 
sJequinu i ho sosn Zuisy 7 99118 91n}99] | B jou siseq} | -10 [euol}eU | Jooyos Jooyos 
je30y, | rr 139730 | -l9apy 00g ouIzese Jo yoeedg | ‘soded a}enpelsy | 10 9}¥}S 10} | 9}¥}s UO [B20] uo 
Te0L | | a8ea1]/09 yodey uodey yodey 
‘ . 
6761 ‘HOUVW AG GALAIAWOD ANV 82-l76l ONINNG NNOAG SAIGNLS AYIVNNOILSANO ILI AO AAVW SASN—O!l ATAVL 
atte Peng on SM (AUS FR RE Ta PETRI pis bites det Scala en 
~ 





f 
f 
q 








The reply received is also affected slightly 
by the character of information requested. The 
71 questionnaires asking for objective data 
only, received a median reply of 74 percent, 
the 51 asking for both objective and subjective 
data received a 65 percent reply, and the 14 
asking wholly for subjective material received 
a 61 percent reply. 

Availability of questionnaire results—In 
March, 1929, information was sought as to the 
availability and use made of the tabulations 
and conclusions based on the 267 question- 
naires of 1927-28. Information on these points 
was obtained for 203 of the 267. The studies 
from these 203 questionnaires were in the fol- 
lowing conditions: 


16 or 7.9 percent—not tabulated or used 

16 or 7.9 percent—tabulation still in progress 
18 or 8.9 percent—in long hand only 

56 or 27.6 percent—typewritten 

47 or 23.1 percent—mimeographed 

50 or 24.6 percent—printed or to be printed 


203 ~=100.0 percent 





The foregoing tabulation shows that a year 
after most of the 203 questionnaires had been 
sent out, only 97 or less than half of them had 
been put in mimeographed or printed form so 
that the results could be generally circulated. 
It also appears that at least 16, and possibly 32 
of the 203 were not to be tabulated. It also is 
likely that little progress in tabulating had been 
made by many of the 64 of the 267 who would 
not report on this matter. 

This evidence seems to indicate that at least 
half of the questionnaires which reach the of- 
fices of superintendents are not put in such form 
that they may be circulated among respondents 
or other interested people. 

Uses made of questionnaire studies—The 
267 questionaires of 1927-28 were studied as 
to what use was made of their results. No re- 
port could be obtained on this point for 64 of 
the 267. The presumption is that many of these 
64 were not put to any significant use. Facts 
were obtained on the remaining 203 of the 267. 
Sixteen of the investigators stated that the 
replies to their questionnaire had not been 
tabulated and that no use had been made of 
them and 16 others reported that nothing had 
been completed by March, 1929. One hundred 





and seventy-one studies had been completed and 
used in various ways, as shown in Table 10. |p 
several cases a questionnaire was used for mor 
than one purpose, consequently the reported 
number of uses exceeds the total number 0; 
questionnaires. Of the 171 completed studie, 
34.5 percent were used in connection with |oca! 
school problems ; 8.8 percent were used in work 
ing on a problem of a state school system; | 4/ 
percent were used in reports of state or na 
tional organizations; 15.2 percent contributed 
toward the preparation of graduate theses; | || 
percent for other papers and reports connected 
with college work; 12.9 percent were used in 
speeches; 19.3 percent were used in magazin 
articles; 1.8 percent were used in books; 3.5 
percent were used in advertising; and 9.4 pe: 
cent were used for other purposes. 

It is evident that no use is made of the results 
of some of the questionnaires which reach the 
superintendent’s office. Effort expended in an 
swering such inquiries is wasted. The use made 
of certain other inquiries is narrow—benefiting 
none but the sender of the inquiry. On the 
other hand a substantial portion of the blanks 
contribute to a variety of valuable outcomes 
The use mentioned most frequently is that ot 
preparing reports on local school problems. 
Forty-three percent of the 203 questionnaires 
contributed to the solution of state and loca! 
school problems. 

It has been stated that the majority of ques- 
tionnaires collect data for college theses or term 
papers. The facts do not support this belief. | 
was found that 15 percent of the 171 ques 
tionnaires were used in theses and 11 percent 
in other college work. Some of this 26 percent 
were also used for other purposes. 

Some of the inquiries were issued by schoo 
officials who deplored the necessity of using « 
questionnaire, but who felt that the need for 
information was imperative. Further collection 
and distribution of information on school prac- 
tice by state and national agencies would re- 
duce the number of such questionnaires. 


The fact that 22 of the questionnaires were 
used in speeches and 33 in magazine articles 
does not mean that they were used for those 
purposes alone. In the majority of cases, ques 
tionnaires used in lectures or articles were als 
used in connection with local problems or * 


[ 36] 


spec 


( 


som 





resp 


his | 





mak 
basi 
one 
prov 
path 
Si 
port 
assel 
the ¢ 
of su 
two 
naire 
were 


smal 





cove! 
porte 
naire 
5 ating 
The 

week 
. tende 
Me tionn 
schoo 
schoo. 
struct 
. deal 1 
Oc 
questi 
tively 





inforn 
a brie 
plicati 
ent qi 





questi 


A 


swers, 
blanks 
Ar 
circula 
point, 
selecte: 
replies 


Tt Mer Mae 





ide 


in 
the 
nks 


nes 
ms. 


ires 


ues- 
erm 
lt 


ues 
rent 


hool 


TS 


tor 
tion 


rac- 


vere 
icles 
hose 

ues- 


also 





erro 








ee 


eee ha ee 





thesis material. The lectures and articles ap- 
pear to be by-products of studies made for 
other reasons. In a few cases, however, the only 
use made of the questionnaire data was in a 
speech or article. 

Of the various uses made of questionnaires, 
some will seem more worthy than others to 
respondents. The investigator who sends with 
his questionnaire a statement of his reasons for 
making the study will give the respondent a 
basis on which to decide whether or not it is 
one which he wishes to support, and will im- 
prove the percent of return from those in sym- 
pathy with the project. 

Summary and conclusions—This section re- 
ports and interprets the results of evidence 
assembled in studying the size and character ot 
the questionnaire problem as it affects the offices 
of superintendents of schools. Over a period of 
two years several hundred different question- 
naires received by the offices of superintendents 
were collected, analysed, and followed up. A 
smaller number of questionnaires was dis- 
covered than was expected. Fourteen offices re- 
ported approximately 370 different question 
naires in each of two years. Only one cooper- 
ating city reported more than 100 in a year. 
The average per city was 45, or about one a 
week. It appears that few offices of superin- 
tendents of schools receive more than two ques- 
tionnaires a week. Half of those which reach 
school offices are sent out by superintendents of 
schools or members of their executive or in- 
structional staffs. Considerably more than half 
deal with administrative topics. 

October to April is the “open season” for 
questionnaires. A majority of them are rela- 
tively short, requesting less than 20 items of 
information, most of which can be supplied by 
a brief, objective answer. The amount of du- 
plication was found to be small. In 267 differ- 
ent questionnaires, asking a total of 13,070 
questions, 26 questions calling for identical an- 
swers, occurred on three or more different 
blanks, 

A reply of 69 percent was received by 136 
circulators of questionnaires, reporting on this 
point, who probably constitute a somewhat 
selected group. A somewhat higher percent of 
replies was usually received on those orginat- 


ing from official sources, issued early in the 
school year, and which were short. 

Less than half of those who send question 
naires to offices of superintendents of schools 
put the results received in such form that they 
circulated to respondents 


can be generally 


Such reports as could be obtained as to use 
made of questionnaires, indicate that 43 per 
cent are employed in the solution of state o1 
local problems of school administration. Ap 
proximately 25 percent are used in the prepara 
tion of graduate theses or similar college work. 
The results from many questionnaires are used 
for several valuable purposes. No use is made 
of the results obtained by a considerable per 
cent. 

The data presented seem to indicate that the 
seriousness of the problem of answering ques 
tionnaires has been somewhat exaggerated. 
This is not to be wondered at. When one is 
weighed down with pressing administrative du 
ties even a few poorly conceived questionnaires 
are apt to disturb one’s equanimity. Even 
though the number of questionaires is some 
what smaller than has sometimes been stated, 
it is not implied that the burden of dealing 
with these inquiries is of little weight. Rather, 
it is of sufficient consequence to suggest that 
particular provision be made for this purpose. 
Such provision should be carefully planned 
along the lines suggested in a subsequent sec 
tion. Even a minimum amount of professional 
and clerical service can accomplish much in 
dealing with questionnaires if it is well directed. 
If this is done most of the waste which now 
enters into the questionnaire method of in- 
vestigation, and there undoubtedly is considet 
able, can be eliminated. 

It is 
offices are inadequately supplied with clerical 


realized that many superintendents’ 
help. One of the best ways of securing such 
help is to convince school boards and com- 
munities that such help is essential to the busi 
nesslike operation of a school office, and to em- 
ploy it for significant purposes when it is 
granted. The prompt and accurate answering 
of worthy questionnaires should be provided 
for by all properly equipped offices. Some pro 
vision for discharging this obligation even 
though it may be small, is eminently justified in 
the light of evidence presented elsewhere in 


[ 37 ] 





ee ee 

















this Bulletin. A general failure to provide for 
the efficient discharge of this duty, would cut 
off at the source one of the springs from which 
is drawn the factual information essential to 





efficient school administration. No single grou 
of school people should be more directly in 
terested in preventing such a calamity tha; 
those engaged in school superintendence. 


A Proposed Plan For the Cooperative Regulation of Questionnaires 


It was stated in an earlier section that the 
solution of the questionnaire problem lies not 
in abolition but in proper regulation through 
voluntary, cooperative action on the part of 
those who send and receive questionnaires. The 
purpose of this chapter is to suggest a plan 
through which such action may effectively 
function. 

As was pointed out above, over 80 percent 
of the questionnaires studied in this investiga- 
tion were originated by superintendents of 
schools and their staffs, college professors, or 
representatives of a few educational associa- 
tions or foundations. These three groups have 
it within their power to correct the undesirable 
features of the questionnaire method of investi- 
gation. This may be done by working along 
the following lines. 

Need of a policy and office plan for handling 
incoming questionnaires—An office routine for 
handling questionnaires will save much time. 
Such a plan is needed in the central offices of 
school systems and in other offices which re- 
ceive questionnaires. Just what the details of 
this plan are is less important than that there 
should be one. Many offices otherwise well or- 
ganized show little efficiency in this regard. 

An effective plan for dealing with incoming 
questionnaires will usually require that one 
person be made responsible for deciding as to 
which ones are to be answered, for securing 
the information requested when the decision is 
to answer, for returning them to the sender, 
and for keeping a copy of data supplied. 

In a small school system the superintendent 
working through a clerk or other assistant will 
necessarily exercise these functions. In a larger 
organization, this responsibility could appro- 
priately be delegated to the director of research 
or some other properly qualified person. 

The definite fixing of responsibility for deal- 
ing with questionnaires should not mean it 
should become a one man affair. Rather it 
should mean that some one is made responsible 


for coordinating and conserving such energy 
as is available for this purpose. When this is 
not done waste is likely to result. In one in 
stance, where the route of a questionnaire 
through the offices of a superintendent’s staff 
was traced, it was found that no less than fiy: 
different offices and eight different people had 
spent some time on the questionnaire, and had 
in each case cleverly “‘passed the buck”’ on to 
the next person. Although in toto more than 
enough time to answer the questionnaire had 
been consumed, it still was only partly done. 
Even more startling was the unanimous deci 
sion on the part of the various people~who had 
a try at answering it, when they were called 
together on the matter, that the questionnaire 
was not worth answering. Poor administra- 
tion is a soft term for such a situation. A better 
way is to make some one person responsible for 
shepherding all questionnaires, and to give him 
proper guidance in the form of abriefstatement 
of the policy of the particular organization as 
it concerns answering questionnaires, and tu 
see that responsible members of the staff un 
derstand the policy. 

When this is done the answering of ques- 
tionnaires becomes a recognized function with 
a dignified status whereby the agency concerned 
makes some contribution to the building up of 
educational knowledge, and in return receives 
dividends of information helpful in shaping 
educational policies and procedures. The pro- 
fessional and clerical service alloted to this 
task may be relatively small, but the allotment 
should be definite and should enjoy a recog: 
nized status. To allow this duty to remain 
everybody’s job is likely to result in the con- 
sumption of much effort with little benefit to 
anyone. 

The person made responsible for dealing 
with questionnaires will need to set up a plan 
for deciding which ones are worthy of answer. 
He will need to have in mind certain criteria 
in reaching prompt and proper decisions on 


[ 38] 


Dos air de PN 


he oh Ace al 


quest 
indic 
m ide 
a 
sored 
14+ o 
trom 
on ps 
sent 
parti 
hood 
pared 
and a 
in sor 
2 


and i: 
policy 








¢ 


Rs of Si 


Ab I APE: ves 





nner erm 





; point. A rough rating plan similar to that 
en on the next page should be useful. The 
estions of this plan may be briefly applied as 


q 


indicated below, proper modifications being 


made to meet the particular situation. 

1. Is the questionnaire adequately spon- 

red?—This question is discussed on page 
14 of this bulletin. In general questionnaires 
from responsible agencies such as those cited 
on page 14 should take precedence over those 
sent out by individuals or by agencies of no 
particular standing. There is a greater likeli- 
hood that the questionnaire has been well pre- 
pared if it comes from a well organized office, 
and a better chance that replies will eventuate 
in something useful to the respondent. 

2. Is the purpose of the study frankly stated, 
and is it one which calls for a reply under the 
policy set up for dealing with questionnaires? 


—This subject is discussed on page 23 of this 


Bulletin. One should be generous in applying 
this criterion. 

Questionnaires which assemble data to be 
used in’ writing graduate theses, and partic- 
ularly those which collect data for “term 
papers” have been generally condemned. The 
objection to the term paper questionnaire is 
based on good grounds. It is an imposition to 
ask busy people to take time to reply to the 
ordinary “half-baked”’ questionnaire sent out 
for this purpose. Such questionnaires usually 
promise benefit to no one but the sender and 
frequently not even to him. 

No such general interdict is justifiable in 
regard to graduate theses. If the question 
naire meets the criterion of the rating plan it 
should be answered. It is doubtless a fact that 
this route to the Ph. D. has been overworked 





FIGURE IV 
An Informal Rating Plan For Questionnaires 
If after inspecting a questionnaire, one may answer yes to each of the following 
questions, it should be answered. If no is the answer to several of the questions the 
questionnaire should be promptly returned to the sender unanswered. For further 
elucidation of these criteria see discussion above. 
1. Is the questionnaire adequately sponsored ?_____________ -. Ye No 
2. Is the purpose of the study frankly stated, and is it one 
which calls for a reply under the policy set up for dealing 
ELE A Tet CRE a Te Yes— No 
3. Is the questionnaire on a worthy educational topic ?___- . Yes— No 
| 4. Is the questionnaire well organized ?__._....____-_.-_--_- Yes- No 
_ 5. Are the questions clearly and briefly worded ?__ _... Yes— No— 
| 6. Can most of the questions be briefly answered with a check 
mark or by a fact or figure, and is the number of questions 
requiring extensive subjective replies kept toa minimum?_. Yes— No 
7. Is the information requested not available elsewhere, and 
obtainable only through questionnaire ?________- —ass Yeao=< No 
. Is the questionnaire set up in proper mechanical form?_... Yes—— No 
9. Are the demands of the questionnaire reasonable?__...... Yes—— No 
10. Is a summary of results or other proper return promised 
EL OR ee a ....{. Yes— No— 

















ti 
4 
7 





and that some questionnaires sent out in con- 
nection with graduate theses have been poorly 
conceived and poorly prepared. On the other 
hand some valuable studies have come from 
this source. Schools of education are recogniz- 
ing the need of regulating this phase of the 
questionnaire problem along the line of the 
suggestions made on page 41. When this is 
done the justification of a general interdict 
against the collection of data for use in ad- 
vanced dissertations largely disappears. 

3. Is the questionnaire on a worthy educa- 
tional topic?—This criterion is discussed at 
length on page 14. If a summary of the results 
of the questionnaire would be valuable to the 
respondent, or to persons other than the sender, 
a reply is justified. One should be generous in 
applying this criterion, particularly if the ques- 
tionnaire fully meets the other nine criteria. 

4. Is the questionnaire well organized?— 
For a detailed discussion of this point see 
pages -15-6 and 20-2. If a questionnaire is 
well organized the fact is usually evident at 
a glance. The major and sub-items are logi- 
cally related and other indications of careful 
planning are evident. 

5. Are the questions clearly and briefly 
worded?—The length of questions will neces- 
sarily differ with the nature of the subject 
being studied. Pages 15-6 and 19-22 of this 
Bulletin will be suggestive in applying this 
criterion. 

6. Can most of the questions be briefly 
answered?—Insofar as possible, questions 
should be answerable with a check mark or 
by a fact or a figure, and the number of ques- 
tions requiring extensive, subjective replies 
should be kept to a minimum. The reasons for 
this criterion are given on pages 18-21 and 36 
of this Bulletin. 

7. Is the information requested unavail- 
able?—Is it obtainable only through ques- 
tionnaire? The justification of this criterion is 
self-evident. It is discussed further on page 16. 

8. Is it set up in proper mechanical form?— 
The items which together constitute good me- 
chanical arrangement are discussed on page 22. 
Among the more important of these are neat- 
ness, easy legibility, sufficient space for answers, 
and proper identification. 


9. Are the demands _ reasonable?—This 






question needs to be answered both with : 
importance of the investigation and the 
spondent’s facilities for answering in mind. 
More specific discussion of this point wil] be 
found on pages 18-20. 

10. Is a proper return promised respond- 
ents?—Questionnaires which benefit no one 
but the sender can seldom be justified. This 
point is further elucidated on page 17. 

It is not suggested that a long and tiresome 
application of these criteria be made. I[t js 
enough to have them in mind, or to have 
page 39 of this bulletin open before one as 
he makes the decision “thumbs up” or “thumbs 
down” on a questionnaire. 

If yes can be answered to all or to most of 
the foregoing questions as a result of a brief 
inspection of a questionnaire, then it deserves 
further attention. If any considerable numbe: 
of the criteria have to be answered with a 
no, then the questionnaire should be 
promptly returned to the sender unanswered. 

The application of this rating plan need not 
be a one man performance. The one responsible 
for handling questionnaires in doubtful cases, 
should consult well-informed associates. For 
example, the advice of the school accountant 
should be enlisted in deciding what question- 
naires on school costs should be answered. 

In the case where it is decided that a ques- 
tionnaire is not to be answered, it should be 
returned to the sender, along with a brief \et- 
ter. It would be well to give the reason why 
the questionnaire was not answered. If this 
were generally done, rather than merely throw- 
ing unanswered blanks in the waste basket, 
the effect would be beneficial all around. 

When the decision is to answer, necessary 
data should be assembled from the best source 
the organization affords. One person should 
not attempt to supply answers to all questions. 
Rather he should be responsible for securing 
the data from the proper source or sources. 
Most of the information asked by proper!) 
prepared inquiries can be supplied by clerks. 
When one person in an organization is re- 
sponsible, the load can be properly dis- 
tributed, and a maximum result secured for « 
minimum of time and energy consumed. 

A copy of the data supplied in response to 
questionnaires should be kept on file. This is 


[40] 





it Bae, 2 








desira 
just V 
data | 
may ] 
Th 
all ay 
numb 
offices 
admir 
of ed 
torege 
ticula 
Th 
questi 
agenc 
tionn: 
volve. 
and e 
in de 
ing ot 
Ne 
naires 
dealin 
a sim 
terial 
on pa 
to all 
Too | 
covert 
growt 
needs 
policie 
in sor 
educa 
agenc 
In 
out ql 
merat 
1. 
a sub 
The | 
will it 
ter q 
there 
Survey} 
of the 
the pr 
> 


to gus 














ON Ainge 20 Se 


0 ree th et 





desirable in case a question later arises as to 
‘ust what information was furnished. Also the 
data assembled in replying to a questionnaire 
may later be needed for local use. 

The foregoing paragraphs apply equally to 
all agencies which receive any considerable 
number of questionnaires, whether it be the 
offices of a superintendent of schools and his 
administrative and supervisory staff, a school 
of education, or other agency. Just how the 
foregoing proposals can be applied in any par- 
ticular situation will depend on .circumstances. 

The policy and office plan for dealing with 
questionnaires must be determined by each 
agency for itself. Merely to storm at ques- 
tionnaires is not to solve the problem they in- 
volve. Thoughtful determination of policy, 
and efficient execution of the same is needed 
in dealing with questionnaires as in discharg- 
ing other duties. 

Need of a policy in sending out question- 
naires—IThe need for a policy and plan for 
dealing with incoming inquiries is matched by 
a similar need in sending them out. The ma- 
terial in the sections of this bulletin beginning 
on pages 14 and 18 are particularly pertinent 
to all agencies which send out questionnaires. 
Too often questionnaires, both as to subjects 
covered and numbers dispatched, are 
growths of individual whims or evanscent 
needs rather than fulfillments of well-rounded 
policies in collecting information. This applies 


out- 


in some measure to school systems, schools of 
education, educational organizations, and other 
agencies which assemble educational data. 

In arriving at a policy and plan in sending 
out questionnaires, agencies such as these enu- 
merated might well take these steps: 

1. Once a year the whole matter should be 
a subject of discussion at a staff conference. 
The result will frequently be a plan which 
will involve the sending out of fewer and bet- 
ter questionnaires. As a basis for discussion 
there should be presented at this conference, a 
survey of the number, character, and outcome 
of those sent out by the organization during 
the previous year. 

2. Some simple machinery should be set up 
to guarantee: 


a. That all questionnaires are well pre 
pared and meet criteria such as those 
in the chapter on Better Preparation 
of Questionnaires. 

b. ‘That too many questionnaires are not 
sent out, and particularly that unneces 


sary questions are avoided. 


c. That a permanent, continuing file of 
all questionnaires sent out is kept in 


one place. 


Just how the foregoing will be worked out 
by any particular agency will depend upon ci: 
cumstances. In some cases the whole arrange 
ment for regulating the outgo from a school 
system, school of education, or educational as 
sociation should be on the informal, voluntary 
basis. In other cases it may need to be a rather 
definite scheme closely administered by the 
chief executive, or a member of his staff to 
which this responsibility is delegated. 

The chapter on Better Preparation of Ques 
tionnaires offers a number of suggestions which 
may well be consulted. Material on page 14 
is particularly pertinent to school systems. 

Questionnaires by college students—It is the 
practice of some departments of education to 
encourage students to send out questionnaires 
in connection with graduate theses and even in 
the preparation of term papers. Figures pre- 
sented elsewhere show that such inquiries con- 
stitute but a minor fraction of those which 
reach superintendents’ offices. But this small 
number has probably aroused more antagonism 
than all other questionnaires taken together. 
Evidence in the files of the Research Division 
shows that many questionnaires sent out by 
students are poorly conceived and miserably 
prepared. Furthermore, experience shows that 
even when the results obtained are actually 
summarized, it is often difficult to get a copy 
of this summary. The facilities which these 
students have are frequently inadequate eithe: 
to the proper preparation of forms or the han 
dling of replies. 

This situation suggests that it would be a 
wise step if responsible colleges and universities 
would make it the rule that aH questionnaires 
be sent out by members of the faculty, even 
though ‘some of the data assembled is used in 


[41] 





SS Ee 5 Ree Loragy 


i 





a a 


=~ <a, 


Te 








theses. This practice would have two advan- 
tages. It would guarantee adequate supervi- 
sion. A faculty member would hesitate to have 
his name and official position attached to poorly 
conceived and poorly executed questionnaires. 
Second, it would eliminate the peculiar irrita- 
tion that many school people feel in being 
“asked by some unknown upstart to write 
his dissertation.” 

Probably the principal reason «why ques- 
tionnaires from graduate students in some uni- 
versities have not been up to standard is the 
unreasonable number of studies assigned to 
faculty members for supervision. In some in- 
stitutions, the first step in guaranteeing ade- 
quate guidance of graduate students should be 
a reduction in the number of investigations as- 
signed to the individual professor for super- 
vision. 

Some institutions are finding it wise to pro- 
vide graduate students with specialized serv- 
ices of both a professional and clerical nature. 
A person expert in the editing of manuscripts 
could greatly improve the clarity and vigor of 
the prose in many dissertations. Students work 
more effectively if they may obtain expert help 
in making tabulations and calculations at a 
reasonable cost. When such facilities are pro- 
vided, the student not only works more effec- 
tively but there is also a partial lifting of the 
burden of supervision expected of the responsi- 
ble faculty member. This leaves him free to 
concentrate his guidance upon the broader 
aspects of the student’s investigation. 

In making the foregoing suggestions, it is 
not implied that the circulation of question- 
naires in school systems by college people is a 
one-sided question. It was fully recognized in 
an earlier section that an important part of 
the knowledge now available for the guidance 
of school people has come from questionnaires 
in the hands of competent investigators, who 
have usually been members of college faculties. 
It is also recognized that practical school work- 
ers, superintendents, supervisors, and teachers 
constantly call upon the colleges for service of 
various kinds. It is little wonder that the col- 
lege professor who donates a considerable por- 
tion of his time and energy in furnishing in- 








formation and guidance to school systems, ;; 
sents their failure to reply to his well-prepared 
questionnaire. 

The recognition of these facts, howeye; 
does not call for a modification of the cont: 
tion that all agencies which send out questi 
naires, colleges as well as others, may wit! 
benefit review their whole policy in this regard 
so that it may be well conceived, and so 
practices which discredit this method of 
quiry may be avoided. 

This section has outlined certain lines | 
voluntary, cooperative action which if followed 
by the various agencies which send out and 1 
ceive educational questionnaires, will eliminate 
many of the abuses which have brought this 
legitimate method of investigation into disre- 
pute. The next paragraphs offer an additional! 
facility aimed at the same result. 

A clearing house for questionnaires—In the 
two years that the Research Division has been 
studying the problem it has virtually become « 
clearing house for educational questionnaires. 

It is proposed that the Division undertake to 
continue this function on a permanent basis. 
In order to accomplish this result it would be 
necessary to have cooperation along these lines: 

1. Those who send out educational question- 
naires should send three complete copies of 
their form and letter of transmittal to the 
Research Division of the National Education 
Association, 1201 16th St., N. W., Washington, 
D. C. 

2. When the results of the study or reports based 
on the questionnaire are completed copies 
should be sent to the Research Division. 

3. When it is not practical to furnish a copy of 
the report, then notice should be sent to the 
Research Division as to where the results of 
the questionnaire are available. 


The questionnaires and reports so received 
will be properly classified, cross indexed, and 
filed. 

It would then be possible for the Division 
to organize a “Questionnaire Service” along 
these lines: 


1. Investigators contemplating the circulation o! 
a questionnaire could be loaned copies of in- 
quiries previously sent out on that topic. 

2. Investigators could be furnished a bibliog 
raphy of completed studies on the topic. 


: nard V, asks this question: ‘‘Would it not be better to 
dial nd faculty ‘itherisation of 2 bam. Peto — students = departments of education rather tha n 
urging that the questionnaires be out tice ett This would be acceptable providing an individua! 
member of the he faculty as well as the aw pe as a whole authorizes the q . The rot weneen is that some compe- 
tent person and organization should feel responsible. Any means which gets this result is acceptable. 


[42] 


of 
n 


By | 


for rej 





3. The results of questionnaire investigation 
could be put into wider use, by annually issu- 
ing a summary of the results of “Question- 
naire Studies” for general circulation, and by 
citing these studies in answering inquiries for 


information. 


In developing the “Questionnaire Service” 
the Research Division: 


1. Will not be able to assist investigators in 
drafting questionnaires, to criticize those 
drafted, nor to aid in the revision of question- 
naires proposed for circulation. These are func- 
tions which can be best exercised by the agency 
sponsoring the questionnaire. 

. Will not be able to sponsor nor circulate ques- 
tionnaires for investigators. The Division al- 
ready is called upon by the various Commit- 
tees and Departments of the National Educa- 
tion Association to circulate all the question- 
naires it can wisely handle. Requests that the 
Division sponsor or circulate questionnaires 
which do not have to do with official projects 
of the National 
not be granted. 


Education Association can- 


By entering into an informal, voluntary plan 
for regulating educational questionnaires along 


the lines suggested above, school systems, 
schools of education, and education associations 
could remove most of the evils and preserve the 
good results which come from circulation of 
these inquiries. 

Summary—This chapter proposes an infor 
mal, voluntary plan for the regulation of ques- 
tionnaires in education. It is suggested that all 
agencies which receive questionnaires develop 
a policy and office plan for quickly identifying 
those worthy of attention and for economically 
providing the information they request. Prac- 
tical means of accomplishing these ends are 
outlined. 

It is also proposed that agencies sending out 
questionnaires develop unified policies which 
will result in the reduction of the number of in- 
quiries sent out and which will guarantee that 
those dispatched are well prepared. Certain 
definite suggestions for the attainment of these 
results are offered. 

A proposal is also made to organize a “Ques- 
tionnaire Service,” for the benefit of those who 
conduct or use the results of questionnaire in- 


vestigations. 








| ee 


ee 


= ula 
nnn 
- OY Be Ae 


Saleen 


PO ee ee 








Some Sources of Information 


In a previous section of this Bulletin, it was 
urged that the decision to issue a questionnaire 
be reached only after a thorough canvass has 
been made of information already available. 
Duplication and waste of effort may thereby 
be avoided. This section calls attention to cer- 
tain, although not all, of the sources of in- 
formation which may be utilized by the in- 
vestigator in making such a canvass. 

1. The Office of Education of the United 
States Department of the Interior \—(For- 
merly the Bureau of Education). This govern- 
mental agency is one of the principal store- 
houses of educational information. A letter ad- 
dressed to The Commissioner of Education, 
Office of Education, United States Depart- 
ment of the Interior, Washington, D. C., will 
usually bring pertinent material in bibliographic 
or other form on any educational question. The 
annual bibliography of research studies in edu- 
cation ? first issued in 1926-27 promises in time 
to furnish a complete and prompt round-up of 
educational investigations both in progress and 
completed. The regular record of current edu- 
cational publications * gives classified and an- 
notated references to certain educational period- 
icals and other publications. The annual Edu- 
cational Directory should be familiar to all 
who have to do with questionnaires.* These are 
but a few of many resources ® which the Office 
of Education has for aiding the investigator. 

2. The National Education Association— 
This professional organization has rapidly de- 
veloped its facilities in recent years and through 
its headquarters library, its Research Division 
which issues the Research Bulletin, its Division 
of Publications which issues the Journal and 
the annual volume of Addresses and Proceed- 
ings, and its various other divisions, is able 
either to furnish or to suggest sources of in- 
formation which may exist on most educational 
topics. The periodic round-up of city school re- 





search studies, completed and in progress,° the 
annual summary of educational legislation 

its questionnaire service and other similar r: 

sources might appropriately be consulted ear!) 
by most investigators. It regularly compiles 
unique information in several fields amony 
which are curriculum reconstruction, the work 
of the principal, state educational legislation, 
teacher salaries and retirement systems. Ad- 
dress the Association at 1201 16th St., Wash 

ington, D. C. 

3. The Education Index—This publication 
may be consulted in well-equipped educations! 
libraries. It was begun in 1929, replacing the 
Loyola Educational Index, which covered 
1928. It indexes more than a hundred educa 
tional periodicals and list books and materia! 
in pamphlet form. The expansion of the cove: 
age of this publication and the cumulative value 
of its issues will rapidly provide an indispens: 
ble tool for educational research workers. (H. 
W. Wilson Co., 958 University Ave., New 
York City.) 

4. Readers’ Guide to Periodical Literatur: 
—Indexes, among other magazines, ten we'll! 
known educational magazines, the bulletins of 
the United States Office of Education, and the 
Proceedings of the National Education Asso 
ciation. Available at libraries. Published 
monthly ; began in 1900. (H. W. Wilson Co., 
958 University Ave., New York City.) 

5. International Index to Periodicals 
(Formerly the Readers’ Guide Supplement ) 
Indexed through 1928 nearly a score of tlic 
technical and specialized educational maga- 
zines, and a small number of the foreign 
periodicals emphasizing education. Published 
five times yearly; began in 1907. The educa- 
tional magazines were transferred to the Edu- 
cation Index on January 1, 1929. (H. W. 
Wilson Co., 958 University Ave., New York 
City.) 


1“*United States Office of Education.” Journal of the National Education Association 18: 241; November, 1929. 
2 United States genes the interios Bureau of Education. anv of Research "Studies in Education, 1926-2 


Bulletin No. 22, 1928. 
+ United States 
23, 1928. Government Printing Office, W: 


mn, D. C. 162 
mt of the Interior, Bureau — on. Record of Current Educational Publications. Bulletin No 


4 United States t of the Interior, Bureau of Tdueation Bulletin (1929) No. 1. Educational Directory: 1929. 


FY and 
Department TT Interior, Bureau of Education iM palicie 'N 
N tion Associa tional 


Educa Educa 

April, 1929. Circular No. 5, 1929. Washington, D. C. 16 p. 
or National Education Association, Research 

Dp. 


eR of 


Secondary Soe en ataea 1920-1925. United States 
, 1926. Washington, D » ©. _- 
. City School Research Studies Completed and in Progre 


Division. State Schoo! Legislation, 1929. Washington, D. C. Mimeographed, 5 


[44] 


é 
Print . 
monthly 
An inde 
hooks ir 
many Pp 
ports. A 
Co., 95§ 

7. Ps 
raphy of 
nate sub 
Most re 
tains 40 
Princeto 

8. Ti 
sists of ¢ 
in card 
lished n 
began ir 
3441 Ne 

o. ze 
in Educ 
versities 
ports be 
number 
1927. ( 
Illinois, 

10. 7 
1918-19. 
reports 
periodic: 
theses. ( 
others, fp 
Illinois, 
type of 
sional is: 
which a 

11. E 
Sources 
Adminis 
Carter , 
Bureau 
Columbi 
attentior 
informat 
alphabet 

je 
number 
abstracts 
of the p 
classifica 





6. The United States Catalog: Books in 
Print January 1, 1928—Supplemented by 
monthly issues of the Cumulative Book Index. 
An index, by author, title, and subject, of ail 
books in print in the United States; includes 
many pamphlets and local governmental re- 
ports. Available at libraries. (H. W. Wilson 
Co., 958 University Ave., New York City.) 

7. Psychological Index—Annual bibliog- 
raphy of the literature of psychology and cog- 
nate subjects in the United States and abroad. 
Most recent volume issued June, 1929, con- 
tains 408 pages. (Psychological Review Co., 
Princeton, N. J.) 

8. The Loyola Educational Digest—Con- 
sists of digests of articles and bulletins, issued 
in card form, adapted for library filing. Pub- 
lished monthly, except in July and August; 
began in 1924. (Loyola Educational Digest, 
3441 North Ashland Ave., Chicago, III.) 

9. Titles of Masters’ and Doctors’ Theses 
in Education Accepted by Colleges and Uni- 
versities in the United States—A series of re- 
ports begun in 1917. Issued biennially; latest 
number covers the period ending October 15, 
1927. (College of Education, University of 
Illinois, Urbana, III.) 

10. Ten Years of Educational Research, 
1918-1927—A bibliography of nearly 3,700 
reports of educational research, including 
periodical references, reports, 
theses. Compiled by Walter S. Monroe and 
others, published in 1928 by the University of 
Illinois, Urbana, Ill. This is illustrative of the 
type of valuable reference material of occa- 


and_ doctors’ 


sional issuance by various educational agencies 
which a search will reveal. 

11. Educational Research: Suggestions and 
Sources of Data with Specific Reference to 
Administration—A 77-page pamphlet by 
Carter Alexander, published in 1929 by the 
Bureau of Publications of Teachers College, 
Columbia University, New York City. Calls 
attention to bibliographies and other sources of 
information on some 300 individual items, 
alphabetically arranged. 

12. Teachers Journal and Abstract—Each 
number contains from sixty to eighty signed 
abstracts of articles in educational magazines 
of the previous month. A consistent plan of 
classification is followed. Published monthly, 


Colorado 
Teachers College, Greeley, Colo. 
13. Continuing Studies—In a 


except July and August, by State 


few mayor 
fields of the school curriculum a definite plan 
ot summarizing research has been developed. 
Investigators in the subject fields mentioned 
below will do well to consult the following 
sources of information before undertaking new 
studies. 

a. Arithmetic. Investigations in the field of 
arithmetic are summarized annually by Guy ‘T. 
the 
School Journal. These annual studies supple- 


1925, 


Buswell and published in Elementary 


ment a monograph published in and 

taken together present a complete survey of 

research in arithmetic. The first and the most 
recent compilations follow: 

BusWELL, Guy THOMAs, and Jupp, CHARrLes Hus- 
BARD. Summary of Educational 
Relating to Arithmetic. Supplementary 
tional Monographs, No. 30. University of 
cago Press, Chicago, Ill. 1926. 212 p. 

BUSWELL, Guy THOMAS. “Summary of Arithmetic 
Investigations (1928) I-II.” Elementary School 
Journal 29: 691-8, 737-47; May-June, 1929 


Investigations 
Educa 
Chi- 


b. Grammar, Language, and Composition. 
A bibliography of 264 selected titles has been 
classified and reviewed by R. L. Lyman in the 
following monograph, which will doubtless 
be followed by annual supplements: 


LyMAN, R. L. 
to Grammar, Language, and Composition 
plementary Educational Monographs, No. 36 
University of Chicago Press, 1929. 302 p 


Summary of Investigations Relating 


Sup- 


c. Reading. An annual summary of reading 
investigations is published by William Scott 


Gray in the Elementary School Journal. ‘These 
exhaustive 


reports are supplementary to an 

monograph issued in 1925, and titled as fol 

lows: 

Gray, WILLIAM Scotr. Summary of Investigations 
Relating to Reading. Supplementary Educational 
Monographs, No. 28. University of 
Press, Chicago, Ill. 1925. 275 p. 

The 


titles, is listed below: 


Chicago 


most recent survey, containing 10] 


Gray, WILLIAM Scott. “Summary of Reading In- 
vestigations (July 1, 1927 to June 30, 1928) 
I-II.” Elementary School Journal 29: 443-57, 
496-509; April, March, 1929. 


d. Science. Digests of 70 learning and cur- 


[45] 











ricular studies in science published prior to 
1925 have been brought together by Curtis in 
the following book, which is to be brought up 
to date in future volumes: . 


Curtis, Francis Day. Digest of Investigations in 
the Teaching of Science in the Elementary and 
Secondary Schools. P. Blakiston’s Sons and Co., 
Philadelphia, Pa., 1926. 341 p. 


e. Secondary-school administration. ‘The 
Department of Secondary-School Principals of 
the National Education Association publishes 
from time to time digests of outstanding re- 
search studies in that field. Their latest com- 
pilation is the following: 


Reavis, Witt1AM C. and Butscu, Russet L. S., 
“Abstracts of Unpublished Masters’ Theses in 
the Field of Secondary-School Administration.” 
Bulletin of the Department of Secondary School 
Principals of the National Education Association, 
Bulletin, No. 24, January, 1929, Cicero, Ill. 202 p. 


f. Social sciences. Abstracts of studies in the 
social sciences were presented by Earle U. 
Rugg in the Third, Fifth, and Sixth Year- 
books of the Department of Superintendence 
of the National Education. The latest com- 
pilation is presented in the following volume: 


Ruocc, Earte Unperwoop, Curriculum Studies in 
the Social Sciences and Citizenship, Colorado 
Teachers College Education Series, No. 3, Colo- 
rado State Teachers College, Greeley, Colorado, 
1928. 214 p. 


14. American Council on Education—This 
agency is able to provide information relating 
to higher education. Below are examples of the 
type of information which the Council makes 
available. Its headquarters are at 26 Jackson 
Place, Washington, D. C. 


Ropertson, Davin A. American Universities and 
Colleges. Published for the American Council on 
Education, by Charles Scribner’s Sons, New 
York. 1928. 884 p. 


15. Professional Organizations—Many edu- 
cational associations in the United States spon- 
sor research. A list of all educational associa- 
tions is published each year in the Educational 
Directory of the U. S. Office of Education. 
The most recent issue of the directory ap- 
peared in May, 1929. It may be obtained only 
through purchase (30 cents a copy) from the 





Superintendent of Documents, Governm 
Printing Office, Washington, D. C. 

The following associations are typical, 
sources of data on subjects frequently cover: 
by questionnaires: 

International Kindergarten Union, 12\)| 
16th Street, N. W., Washington, D. C. 

National Society for the Prevention 9; 
Blindness, 370 Seventh Avenue, New York 
City. 

American Child Health Association, 37)) 
Fifth Avenue, New York City. 

16. National Surveys—Studies which «: 
tempt broadly to survey the situation in certain 
major educational fields are growing in num 
ber and offer an essential background for {v- 
ture investigations. Among these are: 

a. Modern Foreign Language Study. This 
thorough-going investigation, in cooperation 
with the Canadian Committee on Foreign 
Languages, has published a number of re- 
searches and has others in preparation. As one 
of the recent publications, the following bul- 
letin is mentioned : 


Purin, C. M. The Trainina of Teachers of th: 
Modern Foreign Languages. Publications of the 
American and Canadian Committees on Modern 
Languages, Vol. 13. Macmillan Co., New York 
1929. 111 p. 


Inquiries should be addressed to the Modern 
Foreign Language Study, 561 West 116th 
Street, New York City. 

b. The United States Office of Education 
now has under way studies of the Land Grant 
Colleges, of the relation of the federal govern 
ment to education and of secondary education. 


It has also been proposed that this office or- 


ganize a survey of teacher training and schoo! 
finance. 


17. Yearbooks of Various Organizations — 
A rapidly growing number of educational or- 
ganizations issue annual volumes which con- 
tain valuable material. Among these are: 


a. The Departments of Superintendence, of 
Elementary School Principals, of Classroom 
Teachers, etc., of the National Education As- 
sociation, 1201 Sixteenth St., Washington, 
D.C. 

b. The National Society For the Study of 
Education. Public School Publishing Com- 
pany, Bloomington, Illinois. 


[46] 


wpe. - 


Ref 


Fron 
which 
proper 
followit 
notated 
complet 
deal wi 
or resp¢ 


1. ALE: 
licits 
1919 

In 
for 
selec 

2. BAW 
Ind 
ceml 

Ce 
and 
used 
sons 
gard 
ing 

3. Bix 
catio 
Tea 
York 

Li 
the | 

4. Bor 
Que: 
37: ¢ 

Qi 
to q 
scien 
labo: 

5. Buci 
nal 
1926 

U1 
(1) 
ques 
trivi 
assis 





_ The National Society of College Teach- 
.s of Education. University of Chicago Press, 

cago, II. 

d. The Educational Press Association. The 
Fifth Yearbook 1929, lists and classifies sev- 
eral hundred periodicals dealing 
with education. 1201 Sixteenth St., Washing- 
ton, D. C. 


The foregoing sources of information .are 


primarily 


llustrative rather than complete. They are 


enough to support the statement that it 1s 
wise to discover what is already available be 
fore one seeks information in any field. Even 
though agencies such as the United States 
Office of Education or the National Educa 
tion Association may not be able directly to 
supply information on an educational topic 
they may be able to put the investigator in 
touch with an agency which can provide part 
or all of what is desired. 


References on the Use of Questionnaires in Educational Research 


From a much larger number of references 
which directly or indirectly bear upon the 
proper use of questionnaires in education, the 
following have been selected and briefly an- 
notated. Many of those cited are worthy of a 
complete reading, particularly by those who 
deal with questionnaires either as investigators 
or respondents. 


1. ALEXANDER, CARTER. School Statistics and Pub- 
licity. Silver, Burdett and Company, Boston, 
1919. p. 59. 

In general discourages use of questionnaires 
for gathering statistics, because of factors of 
selection, and unreliability of answers. 

. Bawven, W. T. “Searching After the Truth.” 
Industrial Education Magazine, 27: 172-3; De- 
cember, 1925. 

Commends the questionnaire as an important 
and valuable method of investigation, when 
used by responsible agencies. Urges that per- 
sons inform themselves as far as possible re- 
garding the subject of their inquiry before issu- 
ing a questionnaire. 

. Brxter, Harotp Hencn. Check Lists for Edu- 
cational Research. Bureau of Publications, 
Teachers College, Columbia University, New 
York City. 1928. p. 40-5. 

Lists forty-six criteria which should govern 
the preparation and sending of questionnaires. 

. Borinc, E. G. “When and How to Reply to a 
Questionary.” American Journal of Psychology 
37: 632-3, October, 1926. 

Quotes a “code of action” for use in replying 
to questionnaires, drawn up by a group of 
scientists in four of the larger psychological 
laboratories. 

BuckinGcHaM, B. R. “The Questionnaire.” Jour- 
nal of Educational Research 14: 54-8; June, 
1926. 

Urges that the investigator assure himself 
(1) that he cannot collect his data except by 
questionnaire, and (2) that the problem is not 
trivial. States that the recipients owe their 
assistance to the cause of education. 


6. Burk, Freperic. “On a Certain Questionnaire 
School and Society 15: 170-3; February 11, 1922. 
A protest against a particular questionnaire 
seeking data on degrees, teacher load, etc., and 
criticises the educational ideals which prompted 
the questionnaire. 
BUTTERFIELD, E. W. 
of the Dotted Line.” 
1-4; January, 1925. 
Criticises the questionnaire method in con 


“The Plenary 
Educational Review 69 


Inspiration 


ducting the Classical Investigation. 
BUTTERFIELD, E. W. 
the Questionnaire.” School and Society 11: 55-6; 
January 10, 1920. 

Outlines a suggested policy for 


“Professional Ethics and 


replying to 

questionnaires, 

Cuarters, W. W. Curriculum Construction 

Macmillan Co., New York. 1924. p. 133-6. 
Points out five major limitations of the ques 

tionnaire. Recommends preliminary trials of the 

form, the use of check questions, and the sub 
stitution of the oral interview whenever pos 
sible. 

. Crawrorp, CLAaupe C. The Technique of Re 
search in Education. University of Southern 
California, Los Angeles, California. 1928. Chap 
ter X, “Questionnaire Technique,” p. 177-87 

Gives specific suggestions; emphasizes pre 
liminary considerations; points out 
faults, such as ambiguity, and leading questions. 
Douctas, H. R. “The Questionnaire—To Be or 
Not To Be.” School and Society 15: 7-9; 
April 8, 1922. 

A defense of the use of questionnaires, when 


common 


4 


39 


properly prepared, and used sparingly. 
FALLON, JOHN F. “The Questiennaire in Edu- 
cational Research.” Catholic Educational Re 
view 23: 539-45; November, 1926. 

Reviews the history of questionnaires ; digests 
the literature concerning them. Defines ques 
tionnaire as: “a printed form presenting for 
answer a number of questions on a single topic, 
or a group of closely connected topics arranged 
under separate heads.” States that when 
bined. with other methods and used by skilled 
workers the questionnaire is of real service 


com- 


[47 ] 








13. 


15. 





i i 
{ 
{ 
tr 
; 
} 
z 
i 
20. 





. KeLtey, TRUMAN 


GauLT, R. H. “History of Questionnaire in 
Method of Research.” Pedagogical Seminary 
1+: 366-83. September, 1907. 

Mentions first written questionnaire known, 
issued by Statistical Society, of London, in 
1838, on the history of strikes. Refers to work 
of G. Stanley Hall, 1880, in using questionnaires 
to study the mental content of children entering 
school. 


. Gooo, Carrer. How to Do Research in Educa- 


tion. Warwick and York, Inc., Baltimore. 1928. 
p. 133-40. 

Summarizes published discussion of the ques- 
tionnaire method in educational research; 
quotes Buckingham, Wylie, and National Com- 
mittee on Research in Secondary Education. 
HANKINSON, Frank. “The Blight of the Ques- 
tionnaire.” Educational Review. 73: 102-8; 
February, 1927. 

Discusses the ambiguity of questionnaires, 
endorses the policy of not answering them at all. 


. Jones, ARTHUR J., Chairman National Com- 


mittee on Research in Secondary Education. 
An Outline of Methods of Research with Sug- 
gestions for High School Principals and Teach- 
ers. U. S. Department of the Interior, Bureau 
of Education. Bulletin, 1926, No. 24, Govern- 
ment Printing Office, Washington, D. C. 31 p. 

Section V, “The Questionnaire,” gives specific 
directions and suggestions for the use of the 
questionnaire method. 
L. Scientific Method—lIts 
Function in Research in Education. The Ohio 
State University Press, Columbus, Ohio, 1929. 
195 p. 

Lecture II, pages 33 to 83, is particularly 
pertinent. 


. Koos, L. V. The Questionnaire in Education. 


Macmillan Co., New York. 1928. 178 p. 

A comprehensive critique and manual on 
questionnaires. Reports analysis of a large 
sampling of published questionnaire investiga- 
tions; evaluates the questionnaire as a method 
of investigation; and makes recommendations 
as to desirable practices in issuing question- 
naires. Selected bibliography deals with ques- 
tionnaire method, and cites representative stud- 
ies which have employed this method of investi- 
gation. 

Linpsay, E, E. “Questionnaires and Follow-up 
Letters.” Pedagogical Seminary 28: 303-7; Sep- 
tember, 1921. 

Reports experiments in which follow-up letters 
brought in a higher number of replies than the 
original inquiry. 

MANN, Horace. Life and Works of Horace 
Mann. Lee and Shepard, Boston, Massachusetts, 
1891. Vol. III, p. 97-109, 157-67; Vol. IV, p. 
151-71. 

Quotes circular letters and replies received, 
issued in 1841, 1842, and 1847. One was sent to 
employers, on the influence of a public school 
education on the earnings of laborers; one was 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


[48] 


sent to teachers on the moral effects of < 
schools, and one to doctors on the proportion 4; 
disease due to ignorance of the laws of hy 


. MatuHews, C. O. “The Effect of the Ord, ; 


Printed Response Words on an Interest (© 
tionnaire.” Journal of Educational Psycho 
128: 34, February, 1929. 

Reports a tendency to give higher percen: 
affirmative replies to item printed on th: 
treme left position on the page. 


y 


X- 


. “QUESTIONNAIRES AND QUESTIONNAIRES.” 4m») 


ican School Board Journal 71: 74; August, 1925 
Says that the average school board in a ci; 
of 25,000 population receives at least one ques 
tionnaire each day of the school year; empha 
sizes the part played by the graduate student ip 

distributing questionnaires. 

Reever, Warp G. How to Write a Thesis. Pur 
lic School Publishing Company, Bloomington 
Illinois. 1925. p. 34-8. 

Mentions ten essential considerations in 
signing and distributing questionnaires. 

Rucc, Harotp O. Statistical Methods Applied 
to Education. Houghton Mifflin Company, Bos- 
ton. 1917. p. 40-55. 

Contains section on the question-blank method 
of collecting educational data. Classifies ques 
tiennaires by three types, those seeking data 
(1) that can be answered from persona! in 
formation, (2) that can be found in schoo! re 
ords, (3) that represent analyses and judg- 
ments. Gives essential steps and guiding prin 
ciples in use of questionnaire method. 
ScHLUTER, W. C. How to Do Research Work 
Prentice-Hall, Inc., New York. 1926. p. 84-7 

States that a proper use of questionnaires can 

be measured by the number of completed 
turns. Makes suggestions as to preparation of 
questionnaires. 
Symonps, PercivaL M. Measurement in Second 
ary Education. Macmillan Co., New York. 1927 
Chapter 16, “The Measurement of Conduct 
p. 333. 

Refers to work by Voelker, Cudy, Woodworth 
and Raubenheimer in using questionnaires 
study character traits. 

Symonps, PercivAL M. “Method of Investiga 
tion of Study Habits.” School and Society 24 
145-52; July 31, 1926. 

Lists twelve investigations of study habits by 
use of questionnaires; points out weaknesses of 
the method for this purpose. 

Toops, HerpertT A. “Validating the Question- 
naire Method.” Jeurnal of Personnel Researc/ 
2: 153-60; August-September, 1923. 

Describes a study in which a 93 percent re- 

turn to a questionnaire was obtained through 
the use of 5 follow-up letters. 
Toops, Herpert A. “Returns From Follow-up 
Letters to Questionnaires.” Journal of Applied 
Psychology 10: 92-101; March, 1926. 

Reports a 100 percent return on a question- 


| 
| 


nai 
Cul 
oris 
65° 
ten 
mo! 
con 
cha 
up 
30. Tre 
Hot 


met 
31. WE 
cati 
59; 


whi 


ing 
cha 


FORM 











naire, through the use of 6 follow-up letters. 
Cumulative percent of 
original questionnaire, 53%, 
65%, 83%, 88%, 97%, 99%, 
tendency of recipients known to senders to reply 
more promptly than strangers. The essential 
conclusion of the study materially 
changed through the replies obtained by follow- 
up letters. 


response as foilows: 
follow-up letters, 


100%. Observes 


was not 


, 


. Wurrney, F. P. 


Educa 

1924. 
“The results of questionnaires as usually in- 

terpreted hold of 


sometimes 


“Questionnaire Craze 
tional Review 68: 139-40; October, 
tend but to strengthen the 


mediocrity.” Admits that we 


need to know what the other fellow is doing, 
but would discourage the attempt to establish 
norms and averages, through the use of ques 


tionnaire methods. 


30. Trow, W. C. Scientific Method in Education. 3. Wyte, ANpRew T. “To What Extent Max 

Houghton Mifflin Company, Boston, 1925. We Rely Upon the Answers to a School Ques- 

Points out weaknesses of questionnaire tionnaire?” Journal of Educational Method 6: 
method in child psychology. 252-7: February, 1927. 


31. WuippeLe, Guy M. “The Improvement of Edu- Refers to questionnaires to school children. 

cational Research.” School and Society 26: 249- Says answers may be relied upon, provided 

59; August 27, 1927. (1) that questions are plainly worded; (2) 

The writer sets forth some of the ways in that there are a large number of interlocking 

which educational research falls short of be- questions, and (3) that the number of individ 

; ing truly scientific in spirit and practice. States uals questioned is large enough to balance the 
' characteristics of a good questionnaire. errors. 

i 
FORM I.—POSTAL CARD USED BY COOPERATING SCHOOL SYSTEMS IN RE- 


PORTING RECEIPT OF QUESTIONNAIRES. 





A questionnaire has been received at this office from: 


Subject of questionnaire: 


Date of questionnaire: 


This information is furnished to the Research Division of the National 


Education Association through the office of the Superintendent of 
Schools in: 











[ 49} 





FORM II.—FORM USED IN COLLECTING INFORMATION ON PERCENT Or 
RETURN ON QUESTIONNAIRE 








THE NATIONAL EDUCATION ASSOCIATION 
OF THE UNITED STATES 
1201 SIXTEENTH STREET 


WASHINGTON. D.C 


March 29, 1929. 


Study of Questionnaires 


You will recall that last year you sent ue @ copy of your questionnaire on 


Will you now give us the information requested below, concerning the results of 
your questionneire? 


°o 


Have the replies to the above questionnaire been tabulate Yes N 





What uses have been made of the study resulting from the above questionnaire? 
(Please check below) 








Completed In Preparation 

a. Report on a local echool problem........... Rel ees 
bd. Report on problem of state school system... ( ) ae | 
o. Report for state or national organization . ( ) res 
&, Graduate thesia .... 0.2... cc cece eee eee eeee oe od 
e. College paper or report, not a thesis ..... es Pee 
f. Speeoh or lecture ....... 2.6... cece eeeeees ( ) ( ) 
6: a GED soci ccdccccccccccccccvsece ( ) ( ) 
MANNER caacbneesiccwscsyoceocecesi ees PRE, Rlieesta 
i. Gaeavtiaion DRAEAARS ON ees 6abeerssbokbee Ratianad See ee 
j. Other use (please state) 

ee ree ( ) ( ) 


3. Im what mechanical form is the study resulting from the above questionnaire, at 
present? (Please check below) 


@. Handwritten (___) 4. Printed ( ) 


db. Typewritten (__) ee 
0. Mimeographed (___) 


We should be glad to receive a copy of the study resulting from your question- 
naire if available. 


Please return one copy of thie letter, keeping the other for your files. Your 
name is on our mailing list to receive a copy of the report on our study of question- 
naires 


| Leder 


Direotor, Research Division 





{ 50] 








&) 
(as) 


| 


aasag 





SBasooaoaoes 
| 


\6) 


& 





Goose cseSeSsSSGSSSSSS