A virtual rodent predicts the structure of neural activity across behaviours.

Publisher: Nature Publishing Group Country of Publication: England NLM ID: 0410462 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1476-4687 (Electronic) Linking ISSN: 00280836 NLM ISO Abbreviation: Nature Subsets: MEDLINE

Publication: Basingstoke : Nature Publishing Group
Original Publication: London, Macmillan Journals ltd.

Behavior, Animal*/physiology ; Models, Neurological* ; Neural Networks, Computer* ; Virtual Reality*; Animals ; Rats ; Deep Learning ; Motor Cortex/physiology ; Movement/physiology ; Sensorimotor Cortex/physiology ; Female ; Rats, Long-Evans

Animals have exquisite control of their bodies, allowing them to perform a diverse range of behaviours. How such control is implemented by the brain, however, remains unclear. Advancing our understanding requires models that can relate principles of control to the structure of neural activity in behaving animals. Here, to facilitate this, we built a 'virtual rodent', in which an artificial neural network actuates a biomechanically realistic model of the rat ¹ in a physics simulator ² . We used deep reinforcement learning ^3-5 to train the virtual agent to imitate the behaviour of freely moving rats, thus allowing us to compare neural activity recorded in real rats to the network activity of a virtual rodent mimicking their behaviour. We found that neural activity in the sensorimotor striatum and motor cortex was better predicted by the virtual rodent's network activity than by any features of the real rat's movements, consistent with both regions implementing inverse dynamics ⁶ . Furthermore, the network's latent variability predicted the structure of neural variability across behaviours and afforded robustness in a way consistent with the minimal intervention principle of optimal feedback control ⁷ . These results demonstrate how physical simulation of biomechanically realistic virtual animals can help interpret the structure of neural activity across behaviour and relate it to theoretical principles of motor control.
(© 2024. The Author(s), under exclusive licence to Springer Nature Limited.)

Merel, J. et al. Deep neuroethology of a virtual rodent. In Proc. 8th International Conference on Learning Representations 11686–11705 (ICLR, 2020).
Todorov, E., Erez, T. & Tassa, Y. MuJoCo: a physics engine for model-based control. In Proc. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems 5026–5033 (IEEE, 2012).
Hasenclever, L., Pardo, F., Hadsell, R., Heess, N. & Merel, J. CoMic: complementary task learning & mimicry for reusable skills. In Proc. 37th International Conference on Machine Learning (eds Daumé, H. & Singh, A.) 4105–4115 (PMLR, 2020).
Merel, J. et al. Neural probabilistic motor primitives for humanoid control. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).
Peng, X. B., Abbeel, P., Levine, S. & van de Panne, M. DeepMimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph. 37, 1–14 (2018).
Jordan, M. I. in Handbook of Perception and Action, Vol. 2 (ed. Heuer, H.) Ch. 2 (Academic Press, 1996).
Todorov, E. & Jordan, M. I. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226–1235 (2002). (PMID: 1240400810.1038/nn963)
Todorov, E. Direct cortical control of muscle activation in voluntary arm movements: a model. Nat. Neurosci. 3, 391–398 (2000). (PMID: 1072593010.1038/73964)
Lillicrap, T. P. & Scott, S. H. Preference distributions of primary motor cortex neurons reflect control solutions optimized for limb biomechanics. Neuron 77, 168–179 (2013). (PMID: 2331252410.1016/j.neuron.2012.10.041)
Ijspeert, A. J., Crespi, A., Ryczko, D. & Cabelguen, J.-M. From swimming to walking with a salamander robot driven by a spinal cord model. Science 315, 1416–1420 (2007). (PMID: 1734744110.1126/science.1138353)
Kalidindi, H. T. et al. Rotational dynamics in motor cortex are consistent with a feedback controller. eLife 10, e67256 (2021). (PMID: 34730516869184110.7554/eLife.67256)
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982). (PMID: 7143039656436110.1523/JNEUROSCI.02-11-01527.1982)
Evarts, E. V. Relation of pyramidal tract activity to force exerted during voluntary movement. J. Neurophysiol. 31, 14–27 (1968). (PMID: 496661410.1152/jn.1968.31.1.14)
Ashe, J. Force and the motor cortex. Behav. Brain Res. 87, 255–269 (1997). (PMID: 933149410.1016/S0166-4328(97)00752-3)
Kalaska, J. F. From intention to action: motor cortex and the control of reaching movements. Adv. Exp. Med. Biol. 629, 139–178 (2009). (PMID: 1922749910.1007/978-0-387-77064-2_8)
Churchland, M. M. & Shenoy, K. V. Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. J. Neurophysiol. 97, 4235–4257 (2007). (PMID: 1737685410.1152/jn.00095.2007)
Yamins, D. L. K. et al. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl Acad. Sci. USA 111, 8619–8624 (2014). (PMID: 24812127406070710.1073/pnas.1403112111)
Kar, K., Kubilius, J., Schmidt, K., Issa, E. B. & DiCarlo, J. J. Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior. Nat. Neurosci. 22, 974–983 (2019). (PMID: 31036945878511610.1038/s41593-019-0392-5)
Khaligh-Razavi, S.-M. & Kriegeskorte, N. Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol. 10, e1003915 (2014). (PMID: 25375136422266410.1371/journal.pcbi.1003915)
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V. & McDermott, J. H. A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy. Neuron 98, 630–644 (2018). (PMID: 2968153310.1016/j.neuron.2018.03.044)
Wang, P. Y., Sun, Y., Axel, R., Abbott, L. F. & Yang, G. R. Evolving the olfactory system with machine learning. Neuron 109, 3879–3892 (2021). (PMID: 3461909310.1016/j.neuron.2021.09.010)
Singh, S. H., van Breugel, F., Rao, R. P. N. & Brunton, B. W. Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nat. Mach. Intell. 5, 58–70 (2023). (PMID: 378862591060183910.1038/s42256-022-00599-w)
Haesemeyer, M., Schier, A. F. & Engert, F. Convergent temperature representations in artificial and biological neural networks. Neuron 103, 1123–1134.e6 (2019). (PMID: 31376984676337010.1016/j.neuron.2019.07.003)
Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013). (PMID: 24201281412167010.1038/nature12742)
Higgins, I. et al. Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons. Nat. Commun. 12, 6456 (2021). (PMID: 34753913857860110.1038/s41467-021-26751-5)
Banino, A. et al. Vector-based navigation using grid-like representations in artificial agents. Nature 557, 429–433 (2018). (PMID: 2974367010.1038/s41586-018-0102-6)
Cueva, C. J. & Wei, X.-X. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. In Proc. 6th International Conference on Learning Representations (ICLR, 2018).
Grillner, S. et al. in Progress in Brain Research, Vol. 165 (eds Cisek, P. et al.) 221–234 (Elsevier, 2007).
Knüsel, J., Crespi, A., Cabelguen, J.-M., Ijspeert, A. J. & Ryczko, D. Reproducing five motor behaviors in a salamander robot with virtual muscles and a distributed CPG controller regulated by drive signals and proprioceptive feedback. Front. Neurorobot. 14, 604426 (2020). (PMID: 33424576778627110.3389/fnbot.2020.604426)
Michaels, J. A., Schaffelhofer, S., Agudelo-Toro, A. & Scherberger, H. A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping. Proc. Natl Acad. Sci. USA 117, 32124–32135 (2020). (PMID: 33257539774933610.1073/pnas.2005087117)
Sussillo, D., Churchland, M. M., Kaufman, M. T. & Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015). (PMID: 26075643511329710.1038/nn.4042)
Chiel, H. J. & Beer, R. D. The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. Trends Neurosci. 20, 553–557 (1997). (PMID: 941666410.1016/S0166-2236(97)01149-1)
Scott, S. H. & Loeb, G. E. The computation of position sense from spindles in mono- and multiarticular muscles. J. Neurosci. 14, 7529–7540 (1994). (PMID: 7996193657688410.1523/JNEUROSCI.14-12-07529.1994)
Latash, M. L., Scholz, J. P. & Schöner, G. Motor control strategies revealed in the structure of motor variability. Exerc. Sport Sci. Rev. 30, 26–31 (2002). (PMID: 1180049610.1097/00003677-200201000-00006)
Dunn, T. W. et al. Geometric deep learning enables 3D kinematic profiling across species and environments. Nat. Methods 18, 564–573 (2021). (PMID: 33875887853022610.1038/s41592-021-01106-6)
Mimica, B., Dunn, B. A., Tombaz, T., Bojja, V. P. T. N. C. S. & Whitlock, J. R. Efficient cortical coding of 3D posture in freely behaving rats. Science 362, 584–589 (2018). (PMID: 3038557810.1126/science.aau2013)
Markowitz, J. E. et al. The striatum organizes 3D behavior via moment-to-moment action selection. Cell 174, 44–58.e17 (2018). (PMID: 29779950602606510.1016/j.cell.2018.04.019)
Klaus, A. et al. The spatiotemporal organization of the striatum encodes action space. Neuron 95, 1171–1180.e7 (2017). (PMID: 28858619558467310.1016/j.neuron.2017.08.015)
Mimica, B. et al. Behavioral decomposition reveals rich encoding structure employed across neocortex in rats. Nat. Commun. 14, 3947 (2023). (PMID: 374027241031980010.1038/s41467-023-39520-3)
Marshall, J. D. et al. Continuous whole-body 3D kinematic recordings across the rodent behavioral repertoire. Neuron 109, 420–437.e8 (2021). (PMID: 3334044810.1016/j.neuron.2020.11.016)
Berman, G. J., Choi, D. M., Bialek, W. & Shaevitz, J. W. Mapping the stereotyped behaviour of freely moving fruit flies. J. R. Soc. Interface 11, 20140672 (2014). (PMID: 25142523423375310.1098/rsif.2014.0672)
Klibaite, U. et al. Deep phenotyping reveals movement phenotypes in mouse neurodevelopmental models. Mol. Autism 13, 12 (2022). (PMID: 35279205891766010.1186/s13229-022-00492-8)
Pereira, T. D. et al. Fast animal pose estimation using deep neural networks. Nat. Methods 16, 117–125 (2018). (PMID: 30573820689922110.1038/s41592-018-0234-5)
Wu, T., Tassa, Y., Kumar, V., Movellan, J. & Todorov, E. STAC: simultaneous tracking and calibration. In Proc. 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids) 469–476 (IEEE, 2013).
Peng, X. B., Ma, Z., Abbeel, P., Levine, S. & Kanazawa, A. AMP: adversarial motion priors for stylized physics-based character control. ACM Trans. Graph. 40, 1–20 (2021).
Fussell, L., Bergamin, K. & Holden, D. SuperTrack: motion tracking for physically simulated characters using supervised learning. ACM Trans. Graph. 40, 1–13 (2021). (PMID: 10.1145/3478513.3480527)
Dhawale, A. K., Wolff, S. B. E., Ko, R. & Ölveczky, B. P. The basal ganglia control the detailed kinematics of learned motor skills. Nat. Neurosci. 24, 1256–1269 (2021). (PMID: 342673921115219410.1038/s41593-021-00889-3)
Kriegeskorte, N., Mur, M. & Bandettini, P. Representational similarity analysis - connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008). (PMID: 191046702605405)
Jordan, M. I. & Rumelhart, D. E. Internal world models and supervised learning. In Proc. 8th International Workshop on Machine Learning (eds Birnbaum, L. A. & Collins, G. C.) 70–74 (Morgan Kaufmann, 1991).
Nagabandi, A., Kahn, G., Fearing, R. S. & Levine, S. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. Preprint at https://arxiv.org/abs/1708.02596 (2017).
Valero-Cuevas, F. J., Venkadesan, M. & Todorov, E. Structured variability of muscle activations supports the minimal intervention principle of motor control. J. Neurophysiol. 102, 59–68 (2009). (PMID: 19369362271226910.1152/jn.90324.2008)
Diedrichsen, J., Shadmehr, R. & Ivry, R. B. The coordination of movement: optimal feedback control and beyond. Trends Cogn. Sci. 14, 31–39 (2010). (PMID: 2000576710.1016/j.tics.2009.11.004)
Flash, T. & Hogan, N. The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5, 1688–1703 (1985). (PMID: 4020415656511610.1523/JNEUROSCI.05-07-01688.1985)
Harris, C. M. & Wolpert, D. M. Signal-dependent noise determines motor planning. Nature 394, 780–784 (1998). (PMID: 972361610.1038/29528)
Wolpert, D. M. Probabilistic models in human sensorimotor control. Hum. Mov. Sci. 26, 511–524 (2007). (PMID: 17628731263743710.1016/j.humov.2007.05.005)
Lai, L. & Gershman, S. J. in Psychology of Learning and Motivation, Vol. 74 (ed. Federmeier, K. D.) Ch. 5 (Academic Press, 2021).
Ramalingasetty, S. T. et al. A whole-body musculoskeletal model of the mouse IEEE Access 9, 163861–163881 (2021). (PMID: 35211364886548310.1109/ACCESS.2021.3133078)
Golub, M., Chase, S. & Yu, B. Learning an internal dynamics model from control demonstration. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) 606–614 (PMLR, 2013).
Shidara, M., Kawano, K., Gomi, H. & Kawato, M. Inverse-dynamics model eye movement control by Purkinje cells in the cerebellum. Nature 365, 50–52 (1993). (PMID: 836153610.1038/365050a0)
Kawai, R. et al. Motor cortex is required for learning but not for executing a motor skill. Neuron 86, 800–812 (2015). (PMID: 25892304593993410.1016/j.neuron.2015.03.024)
Faisal, A. A., Selen, L. P. J. & Wolpert, D. M. Noise in the nervous system. Nat. Rev. Neurosci. 9, 292–303 (2008). (PMID: 18319728263135110.1038/nrn2258)
Dhawale, A. K. et al. Automated long-term recording and analysis of neural activity in behaving animals. eLife 6, e27702 (2017). (PMID: 28885141561998410.7554/eLife.27702)
Chung, J. E. et al. A fully automated approach to spike sorting. Neuron 95, 1381–1394.e6 (2017). (PMID: 28910621574323610.1016/j.neuron.2017.08.030)
Merel, J. et al. Hierarchical visuomotor control of humanoids. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).
Chentanez, N., Müller, M., Macklin, M., Makoviychuk, V. & Jeschke, S. Physics-based motion capture imitation with deep reinforcement learning. In Proc. 11th Annual International Conference on Motion, Interaction, and Games 1–10 (ACM, 2018).
Abdolmaleki, A. et al. A distributional view on multi-objective policy optimization. In Proc. 37th International Conference on Machine Learning (eds Daumé, H. & Singh, A.) 11–22 (PMLR, 2020).
Francis Song, H. et al. V-MPO: on-policy maximum a posteriori policy optimization for discrete and continuous control. In Proc. 8th International Conference on Learning Representations (2020).
Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (2015).
Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. 30th International Conference on Machine Learning (ICML) (2013).
Seabold, S. & Perktold, J. Statsmodels: econometric and statistical modeling with Python. In Proc. 9th Python in Science Conference (eds van der Walt, S. & Millman, J.) 92–96 (SciPy, 2010); https://doi.org/10.25080/majora-92bf1922-011 .
Diedrichsen, J. et al. Comparing representational geometries using whitened unbiased-distance-matrix similarity. Preprint at https://arxiv.org/abs/2007.02789 (2020).
Schütt, H. H., Kipnis, A. D., Diedrichsen, J. & Kriegeskorte, N. Statistical inference on representational geometries. Preprint at https://arxiv.org/abs/2112.09200 (2021).
Nili, H. et al. A toolbox for representational similarity analysis. PLoS Comput. Biol. 10, e1003553 (2014). (PMID: 24743308399048810.1371/journal.pcbi.1003553)

Date Created: 20240611 Date Completed: 20240814 Latest Revision: 20240816

20240816

10.1038/s41586-024-07633-4

38862024