EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance
Jaeyeon Kim Minjeon Jeon Jaeyoon Jung Sang Hoon Woo Jinjoo Lee

Abstract
In this work, we aim to analyze and optimize the EnCLAP framework, a state-of-the-art model in automated audio captioning. We investigate the impact of modifying the acoustic encoder components, explore pretraining with different dataset scales, and study the effectiveness of a reranking scheme. Through extensive experimentation and quantitative analysis of generated captions, we develop EnCLAP++, an enhanced version that significantly surpasses the original.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| audio-captioning-on-audiocaps | EnCLAP++-large | CIDEr: 0.823 FENSE: 0.665 METEOR: 0.269 SPICE: 0.197 SPIDEr: 0.510 |
| audio-captioning-on-audiocaps | EnCLAP++-base | CIDEr: 0.815 FENSE: 0.661 METEOR: 0.257 SPICE: 0.188 SPIDEr: 0.501 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.