The SOCS project (NSF grant 09-68470) investigates the capabilities and potential of social computational systems (SoCS) in the context of citizen science. Citizen science projects are a form of social-computational system. Whether it be volunteers playing a role in massive, distributed sensing networks exploring the migration of birds, or applying their unique human perceptual skills to searching the skies, human motivation and performance is fundamental to system performance. However, undertaking science through a social computational system brings unique challenges. To understand and address these challenges, this proposal presents a three-phase study of SoCS to support scientific research, grounded in group theory and rooted empirically in case studies and action research. More specifically, the proposal includes case studies of several citizen science projects to establish the nature of the SoCS currently in use, development of SoCS to support different kinds of citizen science projects and evaluation of the impacts of these systems on the outputs and processes of the projects. The system development for this project has resulted in a set of serious games to motivate volunteer participants to work on the classification of images of biological species. The systems can be seen and played at http://citizensort.org/. Research Professor Jun Wang acted as replacement PI on this project.
Some key publications from the project are listed below.
<p>A key problem for crowd-sourcing systems is motivating contributions from participants and ensuring the quality of these contributions. Games have been suggested as a motivational approach to encourage contribution, but attracting participation through game play rather than scientific interest raises concerns about the quality of the data provided, which is particularly important when the data are to be used for scientific research. To assess whether these concerns are justified, we compare the quality of data obtained from two citizen science games, one a “gamified” version of a species classification task and one a fantasy game that used the classification task only as a way to advance in the game play. Surprisingly, though we did observe cheating in the fantasy game, data quality (i.e., classification accuracy) from participants in the two games was not significantly different. As well, the quality of data from short-time contributors was at a usable level of accuracy. These findings suggest that various approaches to gamification can be useful for motivating contributions to citizen science projects.</p>
<p>Forgotten Island, a citizen science video game, is part of an NSF-funded design science research project, Citizen Sort. It is a mechanism to help life scientists classify photographs of living things and a research tool to help HCI and information science scholars explore storytelling, engagement, and the quality of citizenproduced data in the context of citizen science.</p>
<p>Two key problems for crowd-sourcing systems are motivating contributions from participants and ensuring the quality of these contributions. Games have been suggested as a motivational approach to encourage contribution, but attracting participation through game play rather than intrinsic interest raises concerns about the quality of the contributions provided. These concerns are particularly important in the context of citizen science projects, when the contributions are data to be used for scientific research. To assess the validity of concerns about the effects of gaming on data quality, we compare the quality of data obtained from two citizen science games, one a “gamified” version of a species classification task and one a fantasy game that used the classification task only as a way to advance in the game play. Surprisingly, though we did observe cheating in the fantasy game, data quality (i.e., classification accuracy) from participants in the two games was not significantly different. As well, data from short-time contributors was also at a usable level of accuracy. Finally, learning did not seem to affect data quality in our context. These findings suggest that various approaches to gamification can be useful for motivating contributions to citizen science projects.</p>