Jessica Luc and colleagues recently published a paper, ‘Does Tweeting Improve Citations? One-Year Results from the TSSMN Prospective Randomized Trial’ [1], in which the authors report the results of a randomised study of the impact on citation counts of sharing academic papers across twitter.
The authors took 112 primary research articles published between 2017 and 2018 in Annals of Thoracic Surgery and The Journal of Thoracic and Cardiovascular Surgery, and divided this set randomly into two groups that had equal numbers of papers that fell into five broad research categories. One group was shared on social media via being ‘tweeted’ by members the ‘Thoracic Surgery Social Media Network’ to their combined followers, an impressive 52,893 accounts, while the other group was ‘non-tweeted’. After one year of follow-up, the ‘tweeted’ group received more than three-times the number of citations as their non-tweeted counterparts; a statistically significant difference (Tweeted +3.1±2.4 vs. Non-Tweeted +0.7±1.3, p<0.001).
These results were surprising and, generally, warmly received, and are now being shared widely across social media and being picked up by media outlets. Indeed, this is how I encountered the paper late one night; a finding I deeply regret seeing for trying to sleep. On twitter, the response to the article seems mixed. The majority, at least those who are retweeting, seem to see in these results a sign that their time spent on social media is useful in terms of disseminating their publications, and many are encouraging academics who were not on twitter or who did not tweet their papers to get in the game.
For some, however, these findings and the response to them has been deeply unsettling – as I have found them.
Think about it – regardless of the content of papers, sharing them on social media increases their citation impact, at least in the short term. To my eyes, this suggests that (many) scientists are not conducting systematic searches for the material they cite in their papers, but simply cite those that have been serendipitously brought to their attention. If so, the risks this poses for introducing major biases into the literature are severe.
The Luc et al. paper currently contains too little data to understand this. It would help if the authors published a list of the papers they examined for us to understand the content of those papers and the raw citation counts for each paper [2]. However, we know scientists right now are struggling to be systematic in their evidence selection. For example, many studies are emerging that suggest that science is suffering from a ‘positive-outcome’ bias – papers that achieve statistically significant results are typically between 2-3 times better cited than those reporting ‘negative’ results [3], but also ‘negative studies’ are much less likely to be published in high impact journals or even published at all [4]. We know avoiding citation bias is difficult – it requires a systematic search strategy on indexed literature databases – and we know getting scientists to publish negative results is also proving difficult.
Does this quirk of scientific behaviour extend to the content post and shared on twitter? Why wouldn’t it? Do we share content that contradicts our favoured beliefs? Or do we share that which seems to build a wall of authority around our opinions? Do we follow people we disagree with? If we do, do we weigh the content they share in a similarly rigorous and sceptical fashion to that shared by those we like and agree with? Do we share with the same enthusiasm content that we know might contradict our previous statements or enrage our followers, even if it is our own negative studies? Probably not.
Perhaps all of this is unsurprising – we have previously discussed a trend we see as worrying in the scientific literature of reducing the content of papers into short summaries – first the abstract, then the key-points came along, and then findings were being reported in the titles of papers [5]. Maybe we have now entering the era of science in 280 characters, soundbites and spin. I hope not, but suspect we are. As university funders now actively encourage creating and cultivating a social media presence and ‘disseminating’ our research to achieve ‘impact’ and increase ‘visibility’, it looks like this future will be forced upon us whether we like it or not.
I realise my awareness of this paper came from my interactions on twitter – I know about it because I’m on twitter, and I’m writing about it now because of that. I find this unsettling –ware the implications for academia if it is to become increasingly reactionary? While blogs and social media discussion might be an appropriate response, should we allow this influence to creep into our scientific publications and research? Maybe my fears are completely unwarranted. Maybe academics select who to follow and what to ‘like’ carefully, casting a sceptical eye over anything shared. However, I also see the massive number of followers that some academics accumulate, which means that social media inflates the voices of some over others. The consequences of this for science are currently largely unknown, but evidence that what is shared on twitter increases citations counts suggests that twitter is influencing how scientists find and use evidence.
Author | Rhodri Leng 08.07.2020
Notes
1 Luc JGY, Archer MA, Arora RC, et al. (2020). Does Tweeting Improve Citations? One-Year Results from the TSSMN Prospective Randomized Trial. Ann Thorac Surg. S0003-4975(20)30860-2. doi:10.1016/j.athoracsur.2020.04.065
2 It may well the case that the authors plan further follow-up studies, and that releasing this data would bias their results. If no such study is planned, a list of papers should be published to allow other researchers to explore what might have happened here.
3 Duyx B, Urlings MJE, Swaen GMH, et al. (2017). Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol. ;88:92-101. doi:10.1016/j.jclinepi.2017.06.002
4 Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics 90, 891–904. https://doi.org/10.1007/s11192-011-0494-7
5 Leng, G. Leng, RI. (2020). “Ac-Cent-Tchu-Ate the Positive”. Blog post on ‘The Matter of Facts: On Science and its Problems’. Stable URL: https://www.the-matter-of-facts.com/post/ac-cent-tchu-ate-the-positive