Our industry partnership accelerates citation screening in systematic literature reviews by 25% with the help of artificial intelligence

Citation (title and abstract) screening for a systematic literature review may be accelerated by 25% through the application of artificial intelligence (AI) in a novel method developed by Oxford PharmaGenesis in partnership with Vyasa.

The evidence reported in systematic literature reviews derives from articles that have passed a rigorous screening process. According to a recent study, our new, AI-assisted screening method successfully improves efficiency while reducing the screening burden.

Efficiency and rigour: how the method works

Software and analytics provider Vyasa built a deep-learning tool in consultation with HealthScience communications company Oxford PharmaGenesis to assist reviewers in selecting articles for full-text review. This AI tool, called Layar, operates through two methods: ‘named entity recognition’ and ‘keyword search’.

The named entity recognition models built by Vyasa automatically highlight key concepts (such as a disease, gene, protein, organization or location) in the titles and abstracts. The natural language models used by the platform enable keyword searches that draw the user’s attention to the articles whose titles and abstracts contain pre-specified terms.

Crucially, the AI tool leaves inclusion–exclusion decisions in the hands of the reviewer, ensuring that the rigour of reviewing is maintained.

AI-assisted screening: 25% faster while maintaining accuracy

The pilot study of the new method was presented in January at the 2022 European Meeting of the International Society for Medical Publication Professionals. The study aimed to assess whether using the AI tool could improve efficiency in terms of time and accuracy.

During the study, four reviewers screened 300 articles each: half of these articles were screened without AI and half with the help of the Vyasa-developed AI tool.

AI-assisted screening took 25% less time than manual screening (median time taken, 38 vs 51 minutes, respectively), with similar median accuracy.

A follow-up study with more reviewers and articles is underway to corroborate the findings, and it will be presented at the Professional Society for Health Economics and Outcomes Research Europe 2022 meeting in November.

The power of deep learning

Tanina Cadwell, co-author of the pilot study and Solutions Architect at Vyasa, said:

“The ability for deep-learning models to learn languages and understand complex sentence structures unlocks tremendous opportunity for how we interact with our data. In particular, these models accelerate the analysis of large sets of unstructured content, such as published research and scientific articles, which hold highly valuable information but are increasingly difficult to extract insight from.”

This situation makes systematic literature reviews “ideal candidates for the implementation of artificial intelligence”, according to co-author Kim Wager, Scientific Director of Informatics and Data Science at Oxford PharmaGenesis.

Kim expressed his excitement over the partnership with Vyasa “to develop a citation screening tool to improve efficiency, while leaving decision-making in the hands of the reviewer”.

Tanina echoed this sentiment, saying that “Vyasa’s partnership with Oxford PharmaGenesis has improved the accessibility of complex data sets through the power of deep learning”.

New opportunities for AI in HealthScience

This partnership could soon give rise to AI enhancement of the full-paper review and help with the data extraction phases of systematic reviews.

“For example, we soon hope to be able to use artificial intelligence to identify and extract information about patient populations across treatment arms, drug dosages, and key efficacy and safety outcomes,” specified Kim.

Are you interested in learning about the potential of AI to advance medical research and development? Contact the Informatics and Data Science practice at Oxford PharmaGenesis for an exploratory chat.