This SOCS project (NSF 12-11071) examined strategies for dealing with the flood of digital data that confronts researchers. New techniques, tools and strategies for dealing with massive data sets, whether they consist of vast numbers of base-pair sequences or terabytes of data from all-sky astronomical surveys, present an opportunity to establish a 'fourth paradigm' of scientific discovery, but the task is not easy. In many areas of research, the relentless growth of data sets has led to the adoption of increasingly automated and unsupervised methods of classification. In many cases, this has led to degradation in classification quality, with machine learning and computer vision unable to replicate the successes of human pattern recognition. The growth of citizen science on the web has provided a temporary solution to this problem; in particular, the highly successful Galaxy Zoo (Lintott et al. 2008, 2011) and the Zooniverse projects (Smith et al. 2011, Fischer et al. 2011, Davis et al. 2011), which have grown from it and which this proposal takes as its starting point, have demonstrated that it is possible to recruit hundreds of thousands of volunteers to make an authentic contribution to results, boosting human analysis through the collective wisdom of a crowd of classifiers. However, human classifiers alone will not be able to cope with expected flood of data from future scientific instruments. The project was to develop a next-generation socio-computational citizen science platform that combines the efforts of human classifiers with those of computational systems to maximize the efficiency with which human attention can be used. We recognize that to do so requires a thorough understanding of human motivation and learning in this context, and knowledge of how the proposed system will affect these. The project was a partnership between computer and social scientists, addressing research problems both in automated data analysis and social science through systems implementation, alongside field research and experiments with project participants. This project was conducted in collaboration with the Adler Planetarium (Arfon Smith, PI). Carsten Østerlund served as replacement PI at Syracuse University.
Some key publications from the project are listed below.
<p>This study examines the relative efficacy of citizen science recruitment messages appealing to four motivations that were derived from previous research on motives for participation in citizen-science projects. We report on an experiment (N=36,513) that compared the response to email messages designed to appeal to these four motives for participation. We found that the messages appealing to the possibility of contributing to science and learning about science attracted more attention than did one about helping scientists but that one about helping scientists generated more initial contributions. Overall, the message about contributing to science resulted in the largest volume of contributions and joining a community, the lowest. The results should be informative to those managing citizen-science projects.</p>
<p>How online community members learn to become valuable contributors constitutes a long-standing concern of Community & Technology researchers. The literature tends to highlight participants’ access to practice, feedback from experienced members, and relationship building. However, not all crowdsourcing environments offer participants opportunities for access, feedback, and relationship building (e.g., Citizen Science). We study how volunteers learn to participate in a citizen science project, Planet Hunters, through participant observation, interviews, and trace ethnography. Drawing on Sørensen’s sociomaterial theories of presence, we extend the notion of situated learning to include several modes of learning. The empirical findings suggest that volunteers in citizen science engage more than one form of access to practice, feedback, and relationship building. Communal relations characterize only one form of learning. Equally important to their learning are authority–subject and agent-centered forms of access, feedback, and relationship building.</p>
<p>We propose a set of methodological principles and strategies for the use of trace data, i.e., data capturing performances carried out on or via information systems, often at a fine level of detail. Trace data comes with a number of methodological and theoretical challenges associated with the inseparable nature of the social and material. Drawing on Haraway and Barad’s distinctions among refraction, reflection and diffraction, we compare three approaches to trace data analysis. We argue that a diffractive methodology allows us to explore how trace data are not given but created though construction of a research apparatus to study trace data. By focusing on the diffractive ways in which traces ripple through an apparatus, it is possible to explore some of the taken-for-granted, invisible dynamics of sociomateriality. Equally, important this approach allows us to describe what and when distinctions within entwined phenomena emerge in the research process. Empirically, we illustrate the guiding principles and strategies by analyzing trace data from Gravity Spy, a crowdsourced citizen science project on Zooniverse. We conclude by suggesting that a diffractive methodology may help us draw together quantitative and qualitative research practices in new and productive ways that also raises interesting design questions.</p>
<p>Citizen science projects rely on the voluntary contribution of nonscientists to take part in scientific research projects. Projects taking place exclusively over the Internet face significant challenges, chief among them is the attracting and keeping the critical mass of volunteers needed to conduct the work outlined by the science team. The extent to which platforms can design experiences that positively influence volunteers’ motivation can help address the contribution challenges. Consequently, project organizers need to develop strategies to attract new participants and keep existing ones. One strategy to encourage participation is implementing features, which re-enforce motives known to change people’s attitudes towards contributing positively. The literature in psychology noted that novelty is an attribute of objects and environments that occasion curiosity in humans leading to exploratory behaviors, e.g., prolonged engagement with the object or environment. This dissertation described the design, implementation, and evaluation of an experiment conducted in three online citizen science projects. Volunteers received novelty cues when they classified data objects that no other volunteer had previously seen. The hypothesis was that exposure to novelty cues while classifying data positively influences motivational attitudes leading to increased engagement in the classification task and increased retention. The experiments resulted in mixed results. In some projects, novelty cues were universally salient, and in other projects, novelty cues had no significant impact on volunteers’ contribution behaviors. The results, while mixed, are promising since differences in the observed behaviors arise because of individual personality differences and the unique attributes found in each project setting. This research contributes to empirically grounded studies on motivation in citizen science with analyses that produce new insights and questions into the functioning of novelty and its impact on volunteers’ behaviors.</p>
<p>Researchers studying user behaviors in online communities often conduct analyses of events collected in system logs, e.g., a system’s record of a comment post or of a contribution. However, analysis of user behaviors is more difficult if users make contributions without being logged-in (i.e., anonymously). Since a user’s account will not be associated with contributions that user makes anonymously, conclusions about user behaviors that look only at attributed actions might not account for a user’s full experience. To understand the impacts of anonymous contributions on research, we conducted an analysis of system logs containing anonymous activities in two online citizen science projects. By linking anonymous events with user IDs we found that (1) many users contribute anonymously, though with varied patterns of contribution; and (2) including anonymous activities alter conclusions made about users’ experience with the project. These results suggest that researchers of human behaviors in online communities should consider the possible impacts of anonymous interaction on their ability to draw conclusions about user behaviors in these settings.</p>
<p>The explosion of data available from online systems such as social media is creating a wealth of trace data, that is, data that record evidence of human activity. The volume of data available offers great potential to advance social and behavioural science research. However, the data are of a very different kind than more conventional social and behavioural science data, posing challenges to use. This paper adopts a data framework from Earth Observation science and applies it to trace data to identify possible issues in analyzing trace data. Application of the framework also reveals issues for sharing and reusing data.</p>
<p>The paper explores the motivations of volunteers in a large crowd sourcing project and contributes to our understanding of the motivational factors that lead to deeper engagement beyond initial participation. Drawing on the theory of legitimate peripheral participation (LPP) and the literature on motivation in crowd sourcing, we analyze interview and trace data from a large citizen science project. The analyses identify ways in which the technical features of the projects may serve as motivational factors leading participants towards sustained participation. The results suggest volunteers first engage in activities to support knowledge acquisition and later share knowledge with other volunteers and finally increase participation in Talk through a punctuated process of role discovery.</p>
<p>Making the traces of user participation in primary activities visible in online crowdsourced initiatives has been shown to help new users understand the norms of participation but participants do not always have access to others’ work. Through a combination of virtual and trace ethnography we explore how new users in two online citizen science projects engage other traces of activity as a way of compensating. Merging the theory of legitimate peripheral participation with Erickson and Kellogg’s theory of social translucence we introduce the concept of practice proxies; traces of user activities in online environment that act as resources to orient newcomers towards the norms of practice. Our findings suggest that newcomers seek out practice proxies in the social features of the projects that highlight contextualized and specific characteristics of primary work practice.</p>
<p>User-generated content (UGC) projects involve large numbers of mostly unpaid contributors collaborating to create content. Motivation for such contributions has been an active area of research. In prior research, motivation for contribution to UGC has been considered a single, static and individual phenomenon. In this paper, we argue that it is instead three separate but interrelated phenomena. Using the theory of helping behaviour as a framework and integrating social movement theory, we propose a stage theory that distinguishes three separate sets (initial, sustained and meta) of motivations for participation in UGC. We test this theory using a data set from a Wikimedia Editor Survey (Wikimedia Foundation, 2011). The results suggest several opportunities for further refinement of the theory but provide support for the main hypothesis, that different stages of contribution have distinct motives. The theory has implications for both researchers and practitioners who manage UGC projects.</p>
<p>Research on newcomer roles in peer production sites (e.g., Wikipedia) is characterized by a broad and relatively well-articulated set of functionally and culturally recognizable roles. But not all communities come with well-defined roles that newcomers can aspire to occupy. The present study explores activity clusters newcomers create when faced with few recognizable roles to fill and limited access to other participants’ work that serves as an exemplar. Drawing on a mixed method research design, we present findings from an analysis of 1,687 newcomers’ sessions in a citizen science project. Combining session- and individual-level analysis produced three findings (1) newcomers activities manifest a diverse range of session types; (2) Newcomers toggle between light work sessions and more involved types of production or community engagement; (3) an interesting relationship between high-level contributors who do a lot of work but little talk and a small group that does a lot of talk but less work. The former group draws heavily on posts contributed by the latter group. Identifying shifts and regularities in contribution facilitate improved mechanisms for engaging participants and the design of online citizen science communities.</p>
<p>In this paper, we describe the results of an online field experiment examining the impacts of messaging about task novelty on the volume of volunteers’ contributions to an online citizen science project. Encouraging volunteers to provide a little more content as they work is an attractive strategy to increase the community’s output. Prior research found that an important motivation for participation in online citizen science is the wonder of being the first person to observe a particular image. To appeal to this motivation, a pop-up message was added to an online citizen science project that alerted volunteers when they were the first to annotate a particular image. Our analysis reveals that new volunteers who saw these messages increased the volume of annotations they contributed. The results of our study suggest an additional strategy to increase the amount of work volunteers contribute to online communities and citizen science projects specifically.</p>