Rui Ribeiro Joao P. Carvalho Luísa Coheur

Abstract
Recent approaches have attempted to personalize dialogue systems by leveraging profile information into models. However, this knowledge is scarce and difficult to obtain, which makes the extraction/generation of profile information from dialogues a fundamental asset. To surpass this limitation, we introduce the Profile Generation Task (PGTask). We contribute with a new dataset for this problem, comprising profile sentences aligned with related utterances, extracted from a corpus of dialogues. Furthermore, using state-of-the-art methods, we provide a benchmark for profile generation on this novel dataset. Our experiments disclose the challenges of profile generation, and we hope that this introduces a new research direction.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| pgtask-on-pgdataset | gpt2-small | BLEU-1: 61.30 BLEU-2: 32.3 BLEU-3: 20.62 BLEU-4: 9.44 BertScore: 94.39 ROUGE-1: 50.07 ROUGE-2: 28.31 ROUGE-L: 50.00 |
| pgtask-on-pgdataset | gpt2-medium | BLEU-1: 59.31 BLEU-2: 25.94 BLEU-3: 15.3 BLEU-4: 9.17 BertScore: 94.76 ROUGE-1: 46.32 ROUGE-2: 24.14 ROUGE-L: 45.88 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.