Agents in recent machine learning studies, modeling the way children learn open-ended repertoires of skills, need to create and represent goals, select which ones to pursue and learn to achieve them.
We argue that the ability to imagine out-of-distribution goals is key to enable creative discoveries and open-ended learning. Children do so by leveraging the compositionality of language as a tool to imagine descriptions of outcomes they never experienced before, targeting them as goals during play.
Our IMAGINE, an intrinsically motivated deep RL architecture, models this ability. It decomposes between learned goal-achievement reward function and policy relying on deep sets, gated attention and object-centered representations. We introduce the Playground environment and study how this form of goal imagination improves generalization and exploration over agents lacking this capacity.
IMAGINE leverages natural language interactions with a descriptive social partner to explore procedurally-generated scenes and interact with objects. IMAGINE doesn't need externally-provided rewards but learns which behaviors are interesting from language-based interactions with the social partner.