ESD stands for Emotional Speech Database, which is an emotional speech dataset for speech conversion research. The dataset consists of 350 parallel utterances spoken by 10 native English speakers and 10 native Chinese speakers, covering 5 emotion categories (neutral, happy, angry, sad, and surprised). More than 29 hours of speech data were recorded in a controlled acoustic environment. This dataset is suitable for multilingual and cross-lingual emotional speech conversion research.