Nature Index Provides AI-Generated Academic Article Summaries

When publishing an article in a major journal, researchers jump through a near-endless series of hoops to ensure the rigor of their research and the article’s conformity with the style of the journal. The most accessible form of an article is typically the abstract, but even abstracts (which often land between one and three paragraphs) aren’t suitable for publicizing research in the laconic internet age. As a result, many articles are now accompanied by even shorter summaries that can be more easily tweeted and digested. These summaries add yet another hoop for authors and journals to jump through – but what if they didn’t? Now, for the first time, the Nature Index has published summaries of several articles written by AI.

The summaries were shared by Catherine Armitage, chief editor of the Nature Index at Springer Nature, and Markus Kaindl, a computational linguist and senior manager at Springer Nature. The duo selected a trio of articles for testing, pulling from the articles published in Springer Nature publications between 2019 and 2020 that included keywords related to AI, machine learning, or deep learning and which also included mentions of drug discovery or drug design. From that list, they selected three Nature Communications articles that ranked highest in terms of downloads, social media attention, and citations.

Instead of settling on one particular AI summary method – of which there are many – the duo provided five summaries of each article, each from a different source: one from the Allen Institute for Artificial Intelligence’s (AI²) SCITLDR model (using the papers’ abstracts, introductions, and discussions); one from Google’s T5 model (using the abstracts); one from Google’s Pegasus model (using the abstracts); and two from AI²’s SciBERT model (one using the introductions, one using the discussions).

The summaries are imperfect. There are some noticeable typos, and the authors note that the AI-generated text “has not been modified, so does not conform to Nature’s usual editorial standards for grammar and style.” That said, the results are impressive, handily compressing abstracts around 150 words into 15- to 20-word sentences. For example, one article is summarized by: “A generative model that bridges systems biology and molecular design, conditioning a generative adversarial network with transcriptomic data.” (To read the rest of the abstracts and summaries, click here.)

The summaries and discussion were published in the Nature Index’s AI supplement, which aims to draw attention to the utility and challenges involved with the growth of artificial intelligence.

“As AI research output continues to increase, we hope that this supplement can draw attention to some of these advances, but also the ethical and technical challenges that lie ahead,” said David Swinbanks, founder of the Nature Index. “Of particular note is the way that AI is now being used in the scientific publishing industry, where there is much scope to improve the way researchers find and digest content using the power of AI.”