Collective Intelligence: Researchers Propose Rewards for Online Reviews

When using online portals, many people benefit from the reviews of others, even if they remain inactive themselves. Yet it is those who contribute the least to crowdsourcing who could make the greatest contribution to improving a public good. This is the conclusion of a multidisciplinary team of researchers from Hunter College, City University of New York, USA, the University of California, Davis, USA, the Max Planck Institute for Empirical Aesthetics (MPIEA), Frankfurt am Main, Germany, and Princeton University, USA. They recently published their findings in the journal Proceedings of the National Academy of Sciences (PNAS).

The team developed a 3D virtual world where online players had a limited amount of time to collect coins on tropical islands. In the game, ferries that took players from island to island all traveled at different speeds. The game included a rating system so that players could help each other choose the fastest ferry. “You just have to learn over time by exploring and testing which ferries are best,” explains Nori Jacoby, research group leader at the MPIEA and co-senior author on the study. “We made that a collective effort by letting people leave ratings so other people can learn quickly which ferry would be their best choice.”

In this large-scale online experiment, the team collected data from a total of 721 participants playing the game. They found that some rated most or all the ferries, contributing to the public good (in the form of the rating system), while others hardly rated any.

However, when players were awarded coins for providing ratings, the proportion of players who left ratings at all increased from an average of 35 to 70 per cent. Surprisingly, the participants who responded to incentives provided higher quality evaluations by rating more accurately the ferries’ speed. Players who had been providing ratings without a reward tended to rate ferries much less accurately. Combined, the response resulted in significantly more accurate ferry ratings, as a greater percentage of raters and a higher percentage of high-quality raters were included. Jacoby says: “It seems counterintuitive, as most people believe that paying someone to rate will result in low quality ratings. However, we found quite the opposite.”

“It’s a cruel irony that the folks least likely to participate for the collective good are the ones we most need due to their superior skills,” says Dalton Conley, Henry Putnam University Professor in Sociology at Princeton and co-senior author of the paper. First author Ofer Tchernichovski from Hunter College, The City University of New York, and visiting researcher at the MPIEA adds: “Our results demonstrate the promise of large-scale online experiments to contribute to the understanding of complex social structures.”

According to the researchers, giving ratings can be seen as a pro-social action in this game because it benefits everyone. Similar human behaviors in the real world affect online rating systems, forestry management, or even climate change. Co-author Seth Frey from the University of California concludes: “What our results could mean for online platforms or clothing retailers is that if those sites gave rewards for engaging in reviews, they would not only attract more product ratings, but more accurate ratings, which helps everyone.”

Wissenschaftlicher Ansprechpartner:
Max Planck Institute for Empirical Aesthetics
Nori Jacoby: nori.jacoby@ae.mpg.de

Collaborating Research Institutions:
Hunter College, City University of New York: Ofer Tchernichovski (tchernichovski@gmail.com)
University of California, Davis: Seth Frey (sethfrey@ucdavis.edu)
Princeton University: Dalton Conley (daltonclarkconley@gmail.com)

Originalpublikation:
Tchernichovski, O., Frey, S., Jacoby, N., & Conley, D. (2023). Incentivizing Free Riders Improves Collective Intelligence in Social Dilemmas. Proceedings of the National Academy of Sciences (PNAS), 120(46), e2311497120. https://doi.org/10.1073/pnas.2311497120