Respondent Anonymity and Data-Matching Erik Manniche; Donald P. Hayes The Public Opinion Quarterly, Vol. 21, No. 3. (Autumn, 1957), pp. 384-388. Stable URL: http://links.jstor.org/sici?sici=0033-362X%28195723%2921%3A3%3C384%3ARAAD%3E2.0.CO%3B2-A The Public Opinion Quarterly is currently published by Oxford University Press.
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/journals/oup.html. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.
The JSTOR Archive is a trusted digital repository providing for long-term preservation and access to leading academic journals and scholarly literature from around the world. The Archive is supported by libraries, scholarly societies, publishers, and foundations. It is an initiative of JSTOR, a not-for-profit organization with a mission to help the scholarly community take advantage of advances in technology. For more information regarding JSTOR, please contact
[email protected].
http://www.jstor.org Mon Jul 16 17:26:50 2007
384
PUBLIC OPINION QUARTERLY
portion who had been predicted as likely to be well-adjusted. Disproportionately greater numbers of subjects predicted as likely to be poorly adjusted were found among those who had moved for reasons other than to go to college. This trend holds for both sexes.
Respondent Anonymity and Data-Matching* BY ERIKMANNICHE AND DONALD P. HAYES University of Washington
IN
THE use of personal questionnaires for research purposes it is frequently found necessary to request the respondents to write their names on the questionnaires for purposes of making repeated observations or data-matching. In many instances where this is done the researcher, or his critic, feels that the presence of the names on the questionnaires makes the responses less valid. The argument is that a person is less likely to be entirely honest on an identifiable questionnaire than he would be on an anonymous document.' The problem is considered to be aggravated further if, as is often the case, there has been little or no occasion for the establishment of good rapport between researcher and respondent^.^
*The authors express acknowledgement to Professors George A. Lundberg, Otto N. Larsen and Stuart C. Dodd for having read and commented upon this paper. Professor Lundberg initially drew our attention to the problem and has contributed much valuable advice. 1 In attempting to investigate the relationship between responses to questionnaires and anonymity, the authors conducted a simple experiment using questions whose subject matter were likely to be ego-involving to the respondents (e.g. high school grade point average, drinking habits, racial prejudice). Forty-eight subjects were randomly assigned to two treatment groups: the first group maintained strict anonymity; members of the second group were required to identify themselves in writing before they filled out the questionnaire. The analysis of the results revealed significant differences in response between the groups, the anonymous group giving the less socially acceptable responses. "The Effects of Different Conditions of Rapport and Anonymity on Questionnaire Responses," Unpublished manuscript, 1956. Other studies dealing with this problem are: Jack Elinson and Valerie T. Haines, "Role of Anonymity in Attitude Surveys," American Psychologist, 5 (July, 1950), p. 315; John T. Bair and Clinton H. Maag, Attitudes Toward Entering Training As Predictors of Motivation Among Student Aviators, U . S. Naval School of Aviation Medicine Research Report, 1953; W. C. Olsen, "The Waiver of Signature in Personal Reports," lournal of Applied Psychology, 20 (1936), pp. 442-450. 2 The study referred to above further sub-divided each group into two additional groups: one, whose questionnaire was administered by a total stranger, and the other group who received the questionnaire from their regular instructor with whom they had already developed considerable rapport. Significant differences were observed between the responses of the two groups. The group working with the stranger gave the less socially acceptable responses.
LIVING RESEARCH
385
Although researchers may disagree on the extent of the effect of the respondents' fear of possible violation of the promised anonymity, there is probably fairly general consensus on the proposition that-given a set of provocative or embarrassing questions, e.g. on sexual behavior or racial attitudes-it would be preferable either to avoid using respondents' names on questionnaires or to find a standardized method of ensuring that anonymity can and will be preserved. Although the experimental evidence in support of this proposition is meager, and certainly so with respect to what effect (distortion of response) may be expected under different conditions, further evidence of some researchers' belief that respondents will distort the data they provide can be found in the not uncommon practice of utilizing various deceitful procedures, e.g. marking of the questionnaires with invisible ink, et cetera. Clearly, this is an effort to avoid asking respondents to write their names on the document, while preserving facilities for matching questionnaire data with other data. Two major difficulties are involved in the employment of this and similar methods of deception: (1) it is likely to be considered immoral or unethical by both respondents and investigators; (2) social scientists may anticipate a rather unfavorable reaction from the public (potential future respondents) were it to become generally known that such procedures have been or are being used. W e believe that this aspect of the problem is growing increasingly important and acute as social research receives increasing public attention. Although in the large majority of research situations there is no intention of revealing any one person's responses, social scientists are, nevertheless, faced with the fact that such disclosures may be made, whether deliberately or inadvertently, and that the respondents must be convinced that this cannot or will not happen. Thus it seems that there is a need for a procedure or code, as simple and mechanical as possible, which avoids direct use of respondent's name on the questionnaire and which ensures that persons (researchers, clerks) handling the questionnaire or additional identified data cannot know the name of the respondent who filled out a specific questionnaire. Also, persons knowing the code which permits the data-matching should not participate in the administration of the questionnaire or in the processing of data. W e propose the following procedure as a solution. The procedure has been outlined with regard to the common research situation where it is necessary to match data (eg. social attitudes) from a questionnaire with data available in a file (e.g. school record^).^ The procedure makes use of 3As an illustration of studies where the procedure might have been used we cite the following: Charles E. Bowerman and Barbara R. Day, "A Test of the Theory of Complementary Needs As Applied to Couples During Courtship," American Sociological Review, 21 (October, 1956), pp. 602-605; George A. Lundberg and Lenore Dickson, "Inter-Ethnic Relations in a
386
PUBLIC OPINION QUARTERLY
two codes which are co-ordinated by a person not participating in the research proper, and who is also likely to enjoy the confidence of all concerned. PROCEDURE
1. The researcher contacts a colleague and provides him with instructions for gathering background data (e.g. grade-point, IQ) from the file in existence (File A). 2. The colleague selects a clerical worker to whom he gives the instructions (what to do) provided by the researcher. (Strictly speaking, the clerk is dispensable from the point of view of maintenance of anonymity, but it also must be recognized that the role of the colleague calls for a responsible and prestigious person-thus a person not likely to do any great amount of simple clerical work; hence the use of a clerical worker.) 3. Following the instructions, the clerk makes up a separate file (File B, containing data extracted from File A). The clerk staples an index card to each file-form in File B. Each file-form carries only the relevant information concerning the person (in this case: grade-point and IQ), and the index card carries only the person's name. File B is then taken by the clerk to the colleague. 4. The colleague marks each file-form with a code-number; this codenumber is repeated on the index card which the clerk stapled onto the fileform. Having entered identical code-numbers on file-form and card, the colleague now detaches the index cards from the file-forms. The colleague retains the index cards and forwards the numbered file-forms to the researcher. 5. The researcher informs respondents of measures taken to maintain anonymity and then administers the questionnaire. An index card is stapled to each questionnaire. A code-number is found on the index card and identically on the questionnaire proper. The researcher instructs the respondents to write their names on the index card, detach from the questionnaire, and deposit the cards in a sealed box which is taken to the colleague. The respondents now fill out the questionnaires without writing their names on these. 6. The colleague alphabetizes the set of cards from the box and also the set of cards which he himself detached from the file-forms (cf. step 4) ; by matching the two sets he draws up a list of matching numbers. This list is sent to the researcher who can now match the questionnaires and file-forms. Under certain circumstances, the investigator may be forced to collect the supplementary material himself, in which case the following modifications of the above procedure should be observed. High School Population," American lournal of Sociology, 58 (July, 1952), pp. 1-10; Alfred C. Kinsey, et al., Sexual Behavior in the Human Female, Philadelphia: Saunders, 1953.
LIVING RESEARCH
387
A. Each questionnaire is to have stapled to it a card, on which there is space for three sets of numbers and the respondent's name. (i.e. Nam) . The number appearing on the right duplicates the number appearing on the questionnaire. The other two spaces are to be filled in later by the colleague. B. The respondents fill in their names, remove the card from the quesc tionnaire and place it in a sealed box to be opened by the colleague. C. The colleague now places a number of his own choice in one of the two remaining spaces on the card and duplicates this number on the other space. He then tears off that part of the card containing the name and the number closest to the name. T h e investigator is given this set of cards with the understanding that all data to be collated with the questionnaire will have this card attached to them. The colleague retains possession of the code. D. The investigator then transcribes the number shown beside the name on whatever supplemental material has been obtained. E. The supplemental data with their attached cards are taken to the colleague, who removes the cards and turns the code over to the investigator. The investigator may now match the numbers on the two sets of information. CONCLUDING COMMENTS
The main point in the proposed scheme is that no person who has access to the questionnaire knows both halves of the code (actually, two codes) being used. The full code is known only by the colleague who would not usually have access to the data in the questionnaire. The method here outlined is clearly not infallible, and we know of no technique which will do away with corruption. But the method does offer the respondents a measure of assurance that confidential data will not be disclosed, presumably with a consequent reduction of deliberate distortion of information on the part of the respondents. I n practical operation, casual or inadvertent disclosures are eliminated, and the perpetration of simple disc honesty is made more complicated. It is true that the use of this or a similar procedure does complicate the administration of a questionnaire and adds to the burden of work involved. In order to decide whether accepting this additional expense is worth while one must evaluate the likelihood that and extent to which non-anonymous questionnaires will result in distorted responses; simultaneously one should keep in mind the importance or non-importance (i.e. what are the consequences) of obtaining invalid or "half-valid" data. The outlined scheme can be used by any individual or independent research worker, but it also lends itself to easy adaptation for more general use. The method would gain in efficacyand achieve its full psychological poten-
388
PUBLIC OPINION OUARTERLY
tial if it were to become institutionalized. This could be achieved by establishing a specialized office or agency to perform largely those functions which have been assigned to the colleague and his clerk; the current volume of social research is probably sufficient to support several such offices. Several advantages would appear likely to derive from a customary and publicized application of this or a similar arrangement for handling problems of anonymity in personal documents, not the least among them would be the development of regularized standards in research and its ensuing effect upon our human research material. Some such development will probably become more and more necessary as social research matures and social scientists are increasingly called upon to provide knowledge for use on practical problems in the life of society.
Sources of Sampling Bias in Sex Studies BYPURNELL H. BENSON AND EVELYN BENTLEY Drew University
IT
HAS BEEN pointed out that the sampling procedures employed by Kinsey may have introduced biases into his findings.' These procedures provide for the use of respondents willing to be interviewed who are members of organizations included as sampling units. Respondents were obtained from organizations in which the cooperation of one hundred per cent of the members was sought but not always found.2 Kinsey compared the incidence of sexual activities among those from 100 per cent groups with the incidence among those from less than 100 per cent groups, and found minor variations3 H e concluded that his sampling methods do not affect results by more than a few per cent in the incidences reported. 1 W. G. Cochran, Frederick Mosteller, and J. W. Tukey, Statistical Problems of the Kinsey Report, Washington: American Statistical Association, 1954; Paul Wallin, "An Appraisal of Some Methodological Aspects of the Kinsey Report," American Sociological Review, 14 (April 1949), pp. 197-210. A. C. Kinsey, W. B. Pomeroy, and C. E. Martin, Sexual Behavior i n the Human Male, Philadelphia: Saunders, 1948, chapter 3; A. C. Kinsey, W. B. Pomeroy, C. E. Martin, and P. H. Gebhard, Sexual Behavior i n the Human Female, Philadelphia: Saunders, 1953, chapter 2. A. C. Kinsey, W. B. Pomeroy, and C. E. Martin, op. cit., pp. 93-102.
