Kelsey Allen

I am currently a Senior Research Scientist at DeepMind. I completed my PhD at MIT under the supervision of Josh Tenenbaum in the Computational Cognitive Science group and received my B.Sc. from the University of British Columbia in physics.

Email    GitHub    Google Scholar    CV   

I study cognitive science, machine learning, and robotics. I am particularly interested in tool use and tool creation, which can range from every-day challenges, like wielding a broom to get something from under a couch, to grand challenges in science and engineering, like creating wind turbines or rocket ships. At both extremes, people adaptively reshape their environments in ways that expand the scope of what they are individually capable of. My research investigates which computational mechanisms support these complex behaviors in humans, and tries to endow machines with similar abilities to flexibly and appropriately solve new challenges.

* denotes equal contribution.

Selected projects

In this perspectives article, we argue that games provide many advantages over classical paradigms, by being intuitive and fun. We discuss the potentials and pitfalls of games as research platform, and provide guidelines for how to actually use games for your experiments.

We compare people (children and adults) born with congenital limb differences (fewer than two hands at birth) to those born without limb differences on the Virtual Tools game. We find that children and adults born with limb differences solve the game differently from those without. They take fewer attempts to solve each game level, but spend more time thinking between each attempt, relative to people born with two hands.

We develop the first fully unsupervised, learned, 3D simulator without requiring object masks, ground truth physical state information, or point correspondences at training time. The simulator is learned end-to-end using multi-view RGB-Depth sensors, and supports 3D editing, novel view rendering, and stable physics rollouts.

We create a new graph network architecture for modeling rigid body dynamics by using insights from computer graphics for how to represent and resolve collisions. We show that our face interaction graph network (FIGNet) can dramatically improve both the accuracy and efficiency of learned simulators for complex rigid body datasets in simulation, while also being up to 2x more accurate compared to tuned analytic simulators (such as MuJoCo and PyBullet) when fit to real world data.

Using graph neural networks, we learn generic, differentiable physical simulators from offline, task-agnostic datasets. We take gradients through these learned simulators to optimize physical designs for a variety of tasks without further fine-tuning. This method can create 2D containers, funnels, and ramps, 3D landscapes to redirect water, and can be used to design an airfoil as performant as those found with highly engineered solvers.

We present a new domain for testing physical problem solving skills in humans and machines. In these tool-use problems, people demonstrate both `a-ha' insights and local optimization, solving each level in 1-10 attempts. We present a model that uses model-based policy optimization with structured object-oriented priors which mimics human performance. In contrast, a deep reinforcement learning baseline is unable to generalize effectively.

We integrate Task And Motion Planning (TAMP) with primitives that impose stable kinematic constraints or differentiable dynamical and impulse exchange constraints at the path optimization level. This enables the approach to solve a variety of physical puzzles involving tool-use and dynamic interactions, in a similar way to humans on these same puzzles.

We introduce a deep reinforcement learning agent whose policy is represented over the edges of a graph. We demonstrate that this method is able to learn how to glue blocks together to ensure a block tower is stable and generalizes across different sizes of block towers. The relational policy representation obtains super-human performance on this task.

We can learn policies from five or fewer demonstrations that generalize to dramatically different test task instances.

We present infinite mixture prototypes (IMP), which adaptively adjust model capacity by representing classes as sets of clusters and inferring their number. IMP can be trained in fully or semi-supervised settings, and generalizes well to sub-class testing from super-class training (with 10-25% improvements over previous approaches).

Through a set of careful experiments, we show that graph network simulators can model discontinuities common to rigid body dynamics when impulse is instantly exchanged between two objects. Furthermore, we show that graph network simulators can even outperform tuned analytic simulators such as Drake, PyBullet and MuJoCo for real world rigid body dynamics with as few as 16 trajectories of real world data.

We present a simple method for learning a "residual" correction to a policy to improve nondifferentiable policies using model-free deep reinforcement learning.

We present a method for describing and learning transition models in complex uncertain domains using relational rules.

We present a model that learns mode constraints from expert demonstrations. We show that it is data efficient, and that it learns interpretable representations that it can leverage to effectively plan in out-of-distribution environments.

We develop a differentiable physics engine in PyTorch which can be incorporated into standard end-to-end pipelines for model-based learning and control.

We summarize and provide insights on a large body of literature on Graph Neural Networks (GNNs).

We present a model that starts out with a language of low-level physical constraints and, by observing expert demonstrations, builds up a library of high-level concepts that afford planning and action understanding.

We present a simple method for achieving context-dependent filtering of incoming signals in biologically plausible neural networks.

We present a framework for explaining differences between familiar and unfamiliar face processing which combines feedforward neural networks with non-parametric generative models.

We present a coordination game in which three agents must use their knowledge of each others' abilities in order to determine the best action to take. We show that a model which acts to maximize utility under recursive theory of mind, combined with a measure of outcome, best predicts human decisions.

I contributed analyses to determine new limits for particles that might emerge from minimal Z' models.

We analyzed the foraging patterns of red harvester ants and found that ants which had greater numbers of antennal contacts with returning foragers were more likely to leave the nest to forage.

We developed a new set of features based on the automatically generated rhetorical structure of online forum conversations, and show that these features are helpful for detecting disagreement.

Template from here.