ShapeWorld is a novel multimodal deep learning model evaluation method and framework that focuses on generalization capabilities in a formal semantic style. In this framework, artificial data is automatically generated according to predefined specifications. This controlled data generation makes it possible to introduce previously unseen instance configurations during the evaluation process, thus requiring the system to recombine the learned concepts in novel ways.

MIT released this dataset.