A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
Arman Cohan† Franck Dernoncourt* Doo Soon Kim* Trung Bui* Seokhwan Kim* Walter Chang* Nazli Goharian†

Abstract
Neural abstractive summarization models have led to promising results in summarizing relatively short documents. We propose the first model for abstractive summarization of single, longer-form documents (e.g., research papers). Our approach consists of a new hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. Empirical results on two large-scale datasets of scientific papers show that our model significantly outperforms state-of-the-art models.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| text-summarization-on-arxiv | Discourse | ROUGE-1: 35.80 |
| text-summarization-on-pubmed-1 | Discourse | ROUGE-1: 38.93 |
| unsupervised-extractive-summarization-on | SumBasic | ROUGE-1: 29.47 ROUGE-2: 6.95 ROUGE-L: 26.30 |
| unsupervised-extractive-summarization-on | LSA | ROUGE-1: 29.91 ROUGE-2: 7.42 ROUGE-L: 25.67 |
| unsupervised-extractive-summarization-on | LexRank | ROUGE-1: 33.85 ROUGE-2: 10.73 ROUGE-L: 28.99 |
| unsupervised-extractive-summarization-on-1 | LexRank | ROUGE-1: 39.19 ROUGE-2: 13.89 ROUGE-L: 34.59 |
| unsupervised-extractive-summarization-on-1 | SumBasic | ROUGE-1: 37.15 ROUGE-2: 11.36 ROUGE-L: 33.43 |
| unsupervised-extractive-summarization-on-1 | LSA | ROUGE-1: 33.89 ROUGE-2: 9.93 ROUGE-L: 29.70 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.