HyperAI
Command Palette
Search for a command to run...
Cross-modal Generation
Generating content in one modality conditioned on another, such as image captioning, text-to-image, or visual dialogue.
Leaderboard
100 models total
Benchmarks
COCO
ExpansionNet v2
MIMIC-CXR
VizWiz 2020 test-dev
Flickr-30K
COCO-Captions
OSCAR
No-Caps
nocaps in-domain
VinVL (Microsoft Cognitive Services + MSR)
nocaps out-of-domain
PaLI
nocaps entire
nocaps near-domain
GIT2, Single Model
VIST
MSR-VTT
mPLUG-2
AudioCaps
VAST
MSVD
MaMMUT
MS-COCO
NeuSyRE
YouCook2
Univl
Clotho
ZerAuCap
HumanML3D
IU X-Ray
TextCAPS
VisDial v0.9 val
9xFGA (VGG)
CHEXPERT-PLUS
CLEVR-Change
CUB
VQ-Diffusion-F
Spot-the-Diff
VizWiz 2020 test
Gibson
HM3D
nocaps-XD entire
GIT
RefCOCO-g
ActivityNet-Captions
nocaps-val-in-domain
nocaps-val-overall
nocaps-XD in-domain
GIT
nocaps-XD near-domain
GIT
nocaps-XD out-of-domain
GIT
TextCaps 2020
VATEX
VALOR
BoFiT
Image Paragraph Captioning
Multi-Modal-CelebA-HQ
nocaps-val-near-domain
nocaps-val-out-domain
XM3600
ScanRefer
SCICAP
CNN+LSTM (Vision only, First sentence)
CMD-AD
COCO Karpathy
COCO2017 caption
CoMMuTE
Multi-30k
Nymeria
OB-Radix
TV-AD
VggCaps
BOBSL
ChestXray8
Conceptual Captions
ClipCap (MLP + GPT2 tuning)
Daily-PP
DOCCI
Flickr
Flickr30k Captions test
Unified VLP
MS-CXR
Nr3D
Oxford 102 Flowers
RAT-Diffusion
RefCOCO
VinDr-CXR
WIKIWEB2M
Chart-to-Text
ChartMind
ChEBI-20
GIT-Mol
CLEVR-DC
DiagramQG
NBA-Identity
OpenCQA
VC-NBA-2022
Visual Genome
ControlCap
WHOOPS
BDD-X
COCO-2017
Ego4D
EGTEA+ Gaze
Flickr-8k
Image-Chat_AVD
Image-Editing-Request
MSRVTT-CTN
MSVD-CTN
Photo-Chat_AVD
SN-Long
VisualMRC
ActivityNet-Caption
BrainHub
COCO-LN500
HIVAU-70K
How2Sign
Image-Edit-Request
L+M-24
MAD-Eval
ScanQA
VBench
VENUS
AVA-Captions
Birds-to-Words
Clotho v1
Clotho-v2
COCO
Parti Finetuned
COCO-Cap
COCO-EE
COMPOSITION-CAP
EchoVLM dataset
Flickr30k → MSCOCO
HowToNarrate
iGround
L+M
LingoQA
LLaVAR-2-Cap
LLaVAR-2-VQA
MS COCO (Karpathy Split)
MSCOCO → Flickr30k
MUSIC-QA
nocaps val
Prismer
PHOENIX-2014T
R2R
REVERIE
ShareGPT4V & DCI
Stanford Image Paragraph dataset
ViTT
Vid2Seq (VidChapters-7M PT)
A-OKVQA
AVSD-DSTC10
AVSD-DSTC7
AVSD-DSTC8
ChartCap
COCO test
COCO-Caption
COCO Captions test
From Captions to Visual Concepts and Back
COCO-LN
CSL-Daily
DOCCI500
DRAMA
E.E.
Emu-Edit
Flickr30k Entities
Flickr30k-LN
IC
Image Caption
Image-Chat
Instruct-V2Xum
K-VQG
KnowCap Test
Kollenda et al.
LAION-COCO
Parti Finetuned
LEVIR-CC
LLaVA-Med
Localized Narratives
Longitudinal-MIMIC
Med-Trinity
MJHQ-30K
Motion-X
MSCOCO Test
Multilingual-General
Music4way-Any2T
Music4way-MI2T
Music4way-MusicCaps
Music4way-MV2T
MusicCaps
Narratives
Natural Scenes Dataset (NSD)
nuScenes
OK-VQA
OpenImages-I
Parti-Prompts
Reason2Drive
Reasoning
RefCOCO+
RxR
ShareGPT4V, DCI
Shot2Story20K
SoccerNet-v2
Spot-the-Change
Spotlight
TVC
UCCD
UCM-Captions
UniMER-Test
VEGA
VG
VideoXum
VisDial v1.0 test-std
5xFGA + LS*+
VQG-COCO
WTS
YOUCOOKII
ACD
AIC-ICC
AppealImage
BanglaLekhaImageCaptions
CNN + 1D CNN
BCT-CHR
BEATv2
BenSMOT
BlendedSkillTalk
BLiSS
Boxing (BX)
CB-300K
CheXpert
ChinaOpen-1k
GVT
CLD
ClothoCaps
CMER-Bench
Colors
BiLSTMS on color generation
Conceptual
ConvAI2
CoVLA
CRG
CTRG-Brain
CUB-200
CUVA
Data-Comp
DnD Group Gesture dataset
DNICC19k
DocParsing
Dubai-CC
DVQA
EE4D-DescCoach
Ego-Exo4D
Empathetic Dialogues
FetusR
Figure Skating (FS)
Flickr30k Test
Flickr30k-lite
FlickrStyle10K
CapDec
Four Layers
FSVQA
GigaHands
GLaMM
GoodNews
GrowthFetus
HC-STVG
He et al.
Hindi Visual Genome (Challenge Set)
Hindi Visual Genome (Test Set)
HouseTour dataset
IER
J-MID
Kinetics-GEB+
ActBERT-revised
KIT-ML
LeBel's dataset
LN COCO
M3-20M
MAD dataset
MAviS-Bench
Meme-Image-No-Text
MEMECAP
MM-AVS
MMTT
Mol-Instructions
MORE
MS-COCO Karpathy test split
MSVD-Indonesian
Multi-Modal CelebA-HQ
NSD
NuInteract
NWPU-Captions
NYTimes800k
Object Pair 1
Object Pair 2
Object Pair 3
OmniDiff
OmniDrive-nuScenes
Overall
PAd1M
Paragraph Caption
PATENTDESC-355K
PCDes
Peir Gross
BiomedGPT
PETARSeg-11k
Previous Caption
Proprietary
proprietary longitudinal chest CT dataset
PV-VTT
QEVD-bio-fit-coach
QEVD-fit-coach
Real-world audio-visual video evaluation set
Recipe1M
Ref-L4
RQA-70K
RSICD
ScanNet++
ScanQAval
SceneCap
Semantic-HOI
SoccerNet-Caption
Spot-the-Dif
STD
SummScreen3D
SurgVLM
Sydney-Captions
Syned
Syned + EE
T2I-FactualBench
Three Layers
TIB
Trailer-Test
Two Layers
UCFCAP
UMD
UniKnowCap Test
V2C
VALOR-32K
VAR test
VDC
VidChapters-7M
VietPET-RoI
VIEWS
VisDial
VISTA
Visual News
VQA-E
VRG
W3DA
Waymo
Wizard of Wikipedia
WOMD-Reasoning
XMSMO-News
Yo'LLaVA
YouCookII test