Citizen scientists can help ID the progression of bacterial infection in plant cells by determining how “clumpy” plant cell images are.
Explore the microbiome around and inside you with these citizen science projects!
The language on the Clumpy homepage might be considered a challenge for the average citizen scientist: “The model plant-pathogen system comprising the plant Arabidopsis thaliana and the pathogenic bacterium Pseudomonas syringae has been used very effectively to elucidate the nature of the pathogenic interaction.” However, once you get started on this citizen science project, you will soon get a feel for it, and perhaps even enjoy it as I did.
Clumpy is a citizen science project that tests for a bacterial infection in plants. When microbiologists found that organelles in the plant’s photosynthetic cells (i.e. chloroplasts) tended to “clump” together when subjected to bacterial infection, they saw opportunity for a citizen science experiment whereby the public could assist with help in classifying images for ‘clumpiness.’
The Clumpy project came out of observations made by Dr. Littlejohn, a postdoctoral researcher in the Bioscience Department at the University of Exeter; Dr. Murray Grant, a Professor of Plant Molecular Biology; and Dr. John Love, Associate Professor in Plant and Industrial Biotechnology at the University of Exeter. They noticed this unusual phenomenon, which might have important implications for how they understand plant-pathogen interactions. “It was through a conversation with Professor Richard Everson at an Exeter Imaging Network meeting, that the multidisciplinary team got together to set up the online experiment,” said Littlejohn.
The idea to use citizen science to annotate bacterial infections in plants came about in part due to the difficulty in using computational methods alone. Trying to characterize images using abstract notions such as ‘clumpiness’ is an area where humans can easily outperform current computational approaches. In addition, annotation of the images doesn’t require any special expertise, so the problem seemed like an ideal match for citizen science. “We also thought the images themselves were intrinsically interesting, which would help motivate people to provide a wide range of annotations,” said Hugo Hutt, a PhD student in the Department of Physics at the University of Exeter.
Dr. Littlejohn commented, “It would be great to know if it were the plant or the bacterium that initiates the ‘clumping’ of chloroplasts in the leaves. This might help us understand why the chloroplasts clump and which partner in the pathosystem is benefiting from it.” What does this do for society, you might ask? The benefit of a study like this might include fighting diseases in food crops for one.
Verifying the Data
When I first started to use the Clumpy website to classify whether an image was clumpy or not clumpy, I kept second guessing myself, and moved through the selections very deliberately, mulling each one over carefully. They all looked too similar, which got me to thinking about verification of the data. Since all science depends on accurate data, how do they know whether or not answers provided by the average Joe are accurate or not?
Hutt was involved with this aspect of Clumpy—the verification of data:
In 2013 we published an article in Computational Intelligence about this. We tested statistical methods to evaluate the accuracy and reliability of users. For example, to evaluate the accuracy we compared the user annotations with those assigned by an expert. We also measured the degree of consensus among users based on how correlated their annotations were.
The results showed a surprising level of accuracy, which my purely objective test might support—after about 15 attempts I began to recognize ‘clumpiness’ more intuitively, and after just 30 I felt I had it nailed. Hutt and colleagues hope to publish an extended version of the paper this year in a special issue of Soft Computing.
With respect to obtaining a consensus score people were asked to make annotations in one of three paradigms: classification, scoring and ranking. Termed “a web-based citizen science experiment,” Clumpy tasks are evaluated in relation to the accuracy and agreement among the participants using both simulated and real-world data from the experiment. The results show a clear difference in performance between the three tasks, with the ranking task obtaining the highest accuracy and agreement among the participants. That means people like me were capable of producing accurate results when we checked in to Clumpy!
[Note: Read more on how other projects verify crowdsourced data through consensus, and read about one success story about how crowdsourced data was comparable to that produced by leading experts.]
Until now the results have been used to publish more on the computer-collection side, than on the plant biology side, but Littlejohn said, “We are looking to fit this result in with a broader biological study.”
The project has been running since August 2012 and currently houses over 10,000 annotated images. Imagine the cost, time and resources allocated to this process, if citizen scientists had not been involved!
Images: Courtesy of clumpy.ex.ac.uk
Ian Vorster has a MS in Environmental Communications and most recently served as director of communications at the Woods Hole Research Center in Massachusetts. Prior to that he worked in the health communications field. Ian has served as a designer, writer, photographer, editor and project leader in the field of science, and now works freelance in a blend of these roles. You can see more of Ian’s work at www.dragonflyec.com.
