The TrialsTracker Project

Note: PLOS is delighted to once again partner with the Einstein Foundation Award for Promoting Quality in Research with applications and nominations for this year’s awards open until April 30th, 2023 at 10:00 pm UTC.  The awards program honors researchers who reflect rigor, reliability, robustness, and transparency in their work. To get a flavor of what the Einstein Foundation is looking for we have asked last year’s finalists to write about their work. Earlier this month we featured the Open Science Atlas and the Ape Research Index.

Today we hear about from Nicholas DeVito on how he works to improve clinical research transparency with the TrialsTracker project. You can watch a video here and/or read more below.

Clinical trials are essential to evidence generation in medicine. When a trial is conducted everyone from funders, to participants, to practicing clinicians expect, and deserve, the results to be fully reported in a timely manner. This should be uncontroversial but every study on the matter has shown that many trials remain unreported for years after their supposed completion. It is against this backdrop that my colleagues and I created the TrialsTracker project.

This aim of the project is simple: bring public audit and accountability to the trial reporting landscape. The US and the EU have both implemented regulations that require the sponsors of certain clinical trials to register their trials and report results directly to registries within a year of completion. This bypasses common excuses for non-publication regarding journal delays and rejections and removes discretion about if and when results get disseminated. No matter what else a sponsor decides to do with their findings, the public should be guaranteed timely access to summary results from these trials.

The problem, however, was that no one ensured results were being reported. Having rules in place is a major step but without concerted efforts to enforce them they risk becoming ceremonial at best. Neither the US Food and Drug Administration nor the European Medicines Agency seemed particularly interested in publicly (or privately) tracking and pursuing missing results, so we decided to step in and fill the gap.

The requirements of US and EU regulations were carefully translated into simple algorithms that check if a registered trial appears covered under the relevant laws, if it completed and due to report, and if it has results. My software engineering colleagues at the Bennett Institute for Applied Data Science turned this code into applications that pull data from the registries on a regular schedule, process it, and put it on websites detailing who has and has not reported their clinical trials. The FDAAA TrialsTracker and the EU TrialsTracker, both launched in 2018. Users can see an overall picture of reporting, the current status of individual trials, and reporting rates aggregated by sponsor. Sponsors are not only held to public account for their performance but can also use the Trackers as a resource to help manage the reporting of their trial portfolios.

Since we’ve launched, the TrialsTracker project has inspired a full research programme and become a valuable tool for transparency. Our Trackers have been used by parliamentary committees to address trial reporting in the UK, are cited routinely by the press, and relied on by advocates who push for increased attention to research transparency. My doctoral thesis was built on the back of data from the Trackers and the interesting questions they raised about the trial reporting landscape.

The TrialsTrackers show the benefit of building tools that exist outside of simple one-off cross-sectional analyses. We were not the first to examine reporting under the FDA Amendments Act but we aimed to create a living tool that tracks progress in real time. Now, you could get a concrete view of current compliance with a single click, rather than wait for the next paper on the topic.

My colleagues and I are now looking towards the future of the TrialsTracker project. Technical infrastructure requires ongoing maintenance to function. Data fields and flows change, software needs upgrading, and updates are necessary to respond to changing conditions. Additionally, enhancements to the existing trackers can be developed, piloted, and validated. For instance, the trackers could grow to include additional monitoring and notifications for users about trials coming due to report. New European regulations have moved registration and results reporting to a new registry, the Clinical Trials Information System, and a new tracker is needed to stay abreast of transparency developments throughout the EU. This all requires a blend of expert knowledge and technical capacity, which is exactly the environment we aim to foster at the Bennett Institute, and all of this requires resources. In return for this investment, the TrialsTracker will continue to provide an ongoing audit and feedback service on a critical issue in clinical medicine, act as a catalyst for future research, and serve as a conversation starter with key decision makers on how to improve transparency and reporting.