Research

We develop machine learning tools for behavioral and neural data analysis, and conversely learn from the brain to solve challenging machine learning problems.

Research Direction 1

Machine Learning for Behavior Analysis

We strive to develop tools for the analysis of animal behavior. Behavior is a complex reflection of an animal's goals, state and character. Thus, accurately measuring behavior is crucial for advancing basic neuroscience, as well as the study of various neural and psychiatric disorders. However, measuring behavior (from video) is also a challenging computer vision and machine learning problem.

Machine Learning for Behavior Analysis - Figure 2

Machine Learning for Behavior Analysis - Figure 3

Key Tools & Projects

DeepLabCut

Markerless pose estimation for animals

GitHub

DLC2action

Behavior segmentation from pose data

GitHub

AmadeusGPT

Natural language interface for behavior analysis

GitHub

BehaveMAE

Masked autoencoders for hierarchical behavior

GitHub

WildCLIP

Vision-language models for wildlife

GitHub

Research Direction 2

Brain-inspired Motor Skill Learning

Watching any expert athlete, it is apparent that brains have mastered to elegantly control our bodies. This is an astonishing feat, especially considering the inherent challenges of slow hardware and the sensory and motor latencies that impede control. Understanding how the brain achieves skilled behavior is one of the core questions of neuroscience that we tackle through Modeling using Reinforcement Learning, and Control Theory.

Brain-inspired Motor Skill Learning - Figure 2

Key Tools & Projects

DMAP

Distributed morphological attention for locomotion

GitHub

Lattice

Efficient exploration for motor control

arXiv

MyoChallenge 2022

Winning solution (NeurIPS 2022)

GitHub

MyoChallenge 2023

Winning solution (NeurIPS 2023)

GitHub

Research Direction 3

Task-driven Models of Proprioception

We develop normative theories and models for sensorimotor transformations and learning. Work in the past decade has demonstrated that networks trained on object-recognition tasks provide excellent models for the visual system. Yet, for sensorimotor circuits this fruitful approach is less explored, perhaps due to the lack of datasets like ImageNet.

Task-driven Models of Proprioception - Figure 2

Key Tools & Projects

proprioceptive illusion

Deep-learning models of the ascending proprioceptive pathway are subject to illusions

GitHub

Our Mission

Our work strives to understand how the brain creates complex behavior. We develop tools for measuring behavior to achieve that goal, while ensuring they are broadly accessible to the community.

We make models and theories to elucidate how the brain gives rise to behavior with a specific focus on motor control and sensorimotor learning. Measuring behavior is key for assessing and constraining these models.