Trace Methods: Probing the Apparatus

Digital trace data has become increasingly attractive as a source of insight for understanding and evaluating human behavior in computer information systems. Trace data provide rich insights into organizational practices and human behaviors but pose significant methodological challenges due to their fragmented, semi-structured characteristics and the sociotechnical systems that produce them. These challenges require thoughtful reflection by researchers interested in using such methods for their studies. Building on Karen Barad’s concept of an apparatus, we introduce four methodological probes—demarcating phenomena, extending the apparatus, exploring trace diffraction, and identifying differences that matter—as tools for executing reflective trace data studies. We discuss and illustrate how these probes have guided our research’s analysis of human and machine learning in the Gravity Spy citizen science project. We describe how these methodological principles have allowed us to trace the learning dynamics in an increasingly complex web of information systems that record the traces of human behaviors.