This is an update to a research project led by the AI and Data Science Lab of Kobi Gal at Ben-Gurion University.
Many researchers rely on SciStarter to support their research. Find examples here. In 2019, the AI and Data Science Lab of Kobi Gal at Ben-Gurion University began a research collaboration with SciStarter to make it easier for you to find the right project to engage in. Searching through thousands of options can be daunting! Kobi and his team pilot tested a new artificial intelligence (AI) “recommendation system” that used a variety of data to predict which projects you’d prefer. The logic is transparent with messages displayed so you don’t have to guess how the system selected the projects for you. And, the recommendation system is designed to complement, not replace, the popular Project Finder which allows you to choose the criteria you want to search for and select a project.
This post provides an update to the recommendation system. In short, the improved version helps account for something called “popularity bias”.
First, a little history about the project. With thousands of projects listed on SciStarter, a main challenge can be finding the right project, one that really suits your needs and your interests. After meeting at a workshop on the Open Science of Learning in 2018, Kobi Gal, a leading expert in human-centered artificial intelligence from Ben-Gurion University of the Negev at Israel, and Darlene Cavalier, the founder of SciStarter, collaborated (with support from NESTA, a UK-based innovation foundation) to create a smart recommendation system to help SciStarter users find the right project.
The original system matched SciStarter members (people with SciStarter accounts) with projects selected by other users with similar characteristics, based on their profiles and activities. Most of today’s recommendation systems tend to recommend popular projects among the community as they often draw most of the traffic but these recommendations are not always the best fit for the user and can lead to a phenomenon called popularity bias.
As Kobi explained in this Citizen Science podcast covering the original system, “AI is a collection of techniques and technologies that complement human abilities, rather than replace human abilities, and help us to do better jobs, whether as teachers, judges, doctors, or computer users.”
The new system includes a method to leverage less popular projects that haven’t had as much as much promotional exposure on our blogs, social media, newsletter or in the news, or perhaps they are hyper-focused within a narrow location or duration which means they only appear in search results to people searching for projects in that location and during that timeframe. We call these “long-tail projects” (projects that get less exposure) and the new system considers long-tail projects. We expect this will lead to recommendations which are more fair towards less popular projects while still adhering to users’ interests.
As before, a user can opt-out from the recommendation system at any time. If a user opts out, they will get the default, fixed list of popular projects on their SciStarter homepage and on the project page sidebars.
How it works
Recommendation systems (commonly used in e-commerce, in the news, and on social media sites) use algorithms that analyze past behavior to recommend items to users, relying on hundreds of thousands of data instances. The system matches users with items that are liked by similar users, with similarities between users based on their past behavior. For users with significant history with the SciStarter platform, the recommendation engine will make predictions based on the algorithms. For users that are new to the SciStarter platform, the engine will rely on recommending the most popular projects to users.
We mentioned briefly the definition of long tail, which is the group of less popular projects among the pool of projects. Why are those important? As a user, getting recommended popular projects can be nice but may not always fit your interest. These projects might get recommended just for their popularity alone while not really matching users’ interests. The new system aims to minimize the bias towards popular projects and recommend projects to users based on their previous experience on the website. It aims to put user interest as the main focus while considering all available options in a fairer manner.
Assessing success
According to a report from the National Academies of Sciences, Engineering, and Medicine, citizen scientists’ motivations are “strongly affected by personal interests,” and participants who engage in citizen science over a long period of time “have successive opportunities to broaden and deepen their involvement.” Thus, it seems that sustained engagement through the use of intelligent recommendations can improve data quality and scientific outcomes for the projects and the public.
All data collected and analyzed during this experiment on SciStarter is anonymized. This includes all project participation data and clickstream data. The team analyzes the data, comparing the cohorts to examine which types of users have contributed to different projects in terms of the number of visits, saves, length of time spent on the project, and frequency of contributions. The team also conducts surveys of SciStarter users and seeks community feedback (yes, they want to hear from you!).
Our team will continue to share updates with you and with the project owners including insights about other types of projects participants engaged with. Our research aims to yield a reproducible algorithm that citizen science platforms can use for intelligent recommendations, as well as a generalized approach for improving collective intelligence in citizen science by connecting users, data, and AI.
We’d love to hear your opinion on this! Let us know what you think via email at info@scistarter.org.
Thank you to Kobi Gal, Amit Sultan, Avi Segal, and others for their work on this project.