The researchers began by amassing all tweets with vaccination-related keywords and phrases during the 2009 H1N1 pandemic. They then tracked users' sentiments about the H1N1 vaccine. To sort through and categorize the tweets, Salathé's team asked Penn State students to rate a random subset of about 10 percent and them as positive, negative, neutral, or irrelevant. For example, a tweet expressing a desire to get the H1N1 vaccine would be considered positive, while a tweet expressing the belief that the vaccine causes harm would be considered negative. A tweet concerning a different vaccine; for example, the Hepatitis B vaccine, would be considered irrelevant.
Next, the team used the students' ratings to design a computer algorithm for cataloging the remaining 90 percent of the tweets according to the sentiments they expressed. "The human-rated tweets served as a 'learning set' that we used to 'teach' the computer how to rate the tweets accurately," Salathé explained. After the tweets were analyzed by the computer algorithm, the final tally was 318,379 tweets expressing either positive, negative, or neutral sentiments about the H1N1 vaccine.
After categorizing the tweets, Salathé and his team then developed a statistical model with information including the number of microbloggers each Twitter user was following. In addition, the researchers recorded whether those followed microbloggers tended to tweet negatively or positively about the H1N1 vaccine. Also included in the model was the number of the negative or positive tweets each of the followed microbloggers sent out. "How many pro-vaccine or anti-vaccine individuals a Twitter user follows is just one measure. Also important is how many negative or positive tweets each followed microblogger then broadcasts to his followers," Salathé said. "It might be that a Twitter user follows only 5 anti-vaccine microbloggers, but if those 5 microbloggers all send 10 negative tweets per day, that might have an important impact." Other measures included in the statistical model were each Twitter user's number of reciprocal users -- how many pairs of microbloggers were following each other -- and the history of followers' own negative and positive tweets.
The team's first unexpected finding was that exposure to negative sentiment was contagious, while exposure to positive sentiments was not. "Cause and effect are difficult to unravel in data such as these, so we can only speculate about why we saw this happen," Salathé said. "Whatever the reason, the observation is troubling because it suggests that negative opinions on vaccination may spread more easily than positive opinions."
The team's second unexpected finding was that microbloggers with more reciprocal Twitter relationships tended to be influenced differently depending on whether the vaccine sentiments of their connections were positive or negative. "We found that, in reciprocal microblogging relationships, negative sentiments were more socially contagious than positive sentiments," Salathé said. "When a microblogger had a lot of reciprocal Twitter connections with users who expressed anti-vaccine sentiments, he tended to tweet even more anti-vaccine sentiments himself." Interestingly, however, Salathé and his team found that the same did not hold true for microbloggers with reciprocal connections with users who expressed pro-vaccine sentiments; that is, pro-vaccine sentiments did not seem to encourage people to tweet more positive sentiments of their own.
"Our third finding was the most bizarre and perhaps the most discouraging," Salathé said. He and his team looked at the sheer volume of negative or positive tweets followers received -- independent of how many individuals the users followed. "Not surprisingly, we found that a high volume of negative tweets seemed to encourage people to tweet more negatively. But strangely, a high volume of positive tweets seemed to encourage people to tweet more negatively, too," Salathé said. "In other words, pro-vaccine messages seemed to backfire when enough of them were received."
Salathé hopes to design additional Twitter studies to test whether the same effects can be observed for sentiments expressed about other vaccines, as well as about other health issues such as antibiotic usage, dieting, and exercising. "While some of our results from the H1N1 study may seem frustrating, there are silver linings," Salathé said. "First, we have a tried-and-true way to track and analyze the wealth of data out there on Twitter. Second, further studies may reveal why positive messages seem to encourage negative tweeting; perhaps there's something about the manner in which the message is being conveyed. For example, public health officials could use that information to send positive messages in a way that would be more likely to have the intended effect."