Qualitative and Quantitative Evidence Regarding the Intrusiveness of Recording Devices in Naturalistic Research

Critics of naturalistic social science research charge that participant awareness of the existence of a recording device alters the behavior of the research participants, known as the “Hawthorne effect.” This study compares segments of talk in which participants explicitly orient to the recording device against segments of talk without such orientation to determine how and if such orientation alters the behaviors of participants. Conversational data were gathered over a six-year period comprising 64 independent conversations involving 213 subjects. Data were transcribed and coded following the conventions of Conversation Analysis. A total of 18 of the 64 transcribed recordings (28.1%) contained references to the recording device. A total of 284 lines in these eighteen transcribed conversations had references to the recording device and/or the researcher, out of a total of 3,906 lines in the 18 transcriptions (7.3%), or out of a total of 11,675 lines in the entire conversation library (0.02%). Lines from both types of compared segments were coded for turns-at-talk by individual participant. A total of 227 unique turns-at-talk (3.3% of total) in which the recording device was directly addressed were compared to 6,597 unique turns-at-talk in which the recording device was not relevant to the content of the conversation. Results indicate that no statistically significant differences occur between compared segments of talk, thus failing to find evidence to support claims of a “Hawthorne effect” in naturalistic social science research.