Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges

Sanjana Ramprasad, Denis Jered McInerney, Iain Marshall, Byron Wallace

Overview

We present TrialsSummarizer, a prototype for summarizing reports of randomized controlled trials of medical interventions, which are the gold standard for evidence-based care. This is work builds on prior work by Ian Marshall1.

Realizing the aim of evidence-based care has become increasingly difficult owing to the rapid growth of medical literature. The use of summarization methods to provide concise overviews of all evidence relevant to a clinical question can be used to alleviate this difficulty.

However, current neural summarization methods are prone to factual errors rendering them of little use in generating summaries of comparative treatment efficacy. Improving the factuality of neural summarization methods is unlikely to be solved in the near future. In the interim, we present TrialsSummarizer an interactive tool that allows users the means to to assess the validity of outputs. We also provide some degree of controllability that gives users the opportunity to obtain corrected versions of summaries in real time.

Demonstration

Interfaces

We consider two implementations of the summarization module

  1. A vanilla interface based on seq2seq bart model
  2. An exploratory interface based on our proposed model providing transperancy and controllability can be found here

Example Queries

1) Does chloroquine or hydroxychloroquine improve outcomes in COVID-19 infection?

View result on the exploratory interface

2) What is the effect of clopidogrel on mortality rate in patients with acute myocardial infarction?

View result on the exploratory interface

Source Code

https://github.com/sanjanaramprasad/RRnlp

https://github.com/sanjanaramprasad/trialstreamer-demo

References

  1. Marshall, Iain J., et al. “Trialstreamer: A living, automatically updated database of clinical trial reports.” Journal of the American Medical Informatics Association 27.12 (2020): 1903-1912. ↩