We were thrilled last Thursday to read Dr. Alondra Nelson’s United States Office of Science and Technology Policy (OSTP) memorandum on the…
Author: Lauren Cadwallader, Open Research Manager, PLOS
PLOS has released a preprint and supporting data on research conducted to understand the needs and habits of researchers in relation to code sharing and reuse as well as to gather feedback on prototype code notebooks and help determine strategies that publishers could use to increase code sharing.
Our previous research led us to implement a mandatory code sharing policy at PLOS Computational Biology in March 2021 to increase the amount of code shared alongside published articles. As well as exploring policy to support code sharing, we have also been collaborating with NeuroLibre, an initiative of the Canadian Open Neuroscience Platform, to learn more about the potential role of technological solutions for enhancing code sharing. Neurolibre is one of a growing number of interactive or executable technologies for sharing and publishing research, some of which have become integrated with publishers’ workflows.
Early conversations with researchers about the NeuroLibre notebook prototypes suggested we needed to better understand code sharing and reuse from the perspective of both readers and authors. In February 2021 we launched a survey to gather the views of researchers in computational biology and related disciplines. 188 respondents completed the survey.
What did we learn about readers?
Three-quarters of the respondents told us they look at code associated with a research paper at least occasionally, with 39% looking at code frequently or very frequently. Only 6% said they never looked at the associated code, demonstrating that openly available code is valuable to other researchers.
The most common reason (70%) respondents gave for looking at the code was to aid their understanding of the article. Just under half of respondents (48%) want to reuse the code in some way, for example directly reusing the code or reusing selected parts of the code.
A ‘Link to code repository’ (for example Github or Bitbucket) was rated as the most useful method for accessing others’ code and had almost been universally encountered (98%). Aside from ‘Available on request’, respondents rated ‘Link to a website’ and ‘executable code capsule in the research article’ as the least useful methods of accessing code.
Mean usefulness of different methods of code sharing. A score of zero equates to ‘not at all useful’ and 100 equals ‘extremely useful’. The mean values are given above the bars. Error bars show the 95% confidence interval.
What did we learn about authors?
Our survey explored how important certain tasks are to researchers when sharing code, and how satisfied they are with their ability to complete them. Overall, authors of articles with associated code are satisfied with their own ability to carry out many code sharing related tasks. Being able to ensure that ‘Readers can easily run the code in the correct environment’, and ‘The data and code are in the same place’, had the lowest satisfaction scores, implying there may be opportunities for better tools (solutions) to support researchers with these tasks.
The majority of respondents reported that they spend more than one day preparing code for sharing alongside a publication. Using new tools or platforms for preparing and sharing code alongside a publication can require additional costs – in terms of time and effort – but there was no consensus, from our results, on whether researchers might be willing to spend additional time on using a tool, e.g. a notebook, to improve the utility of the code they are sharing.
Technological and cultural solutions
The findings of our survey have shown us that whilst researchers recognise the benefits that using code sharing technology bring, such as sharing data and code in the correct environment and allowing readers to easily change parameters, there may be insufficient incentives for researchers to carry out these tasks routinely when publishing in a journal. Most researchers in our survey appear to be satisfied with the community norm of sharing code via a code repository. While technological solutions could help researchers in other ways, to support our goal of increasing sharing and reuse of code, PLOS Computational Biology’s focus on the “cultural solution” of a mandatory policy seems to be the optimal approach.