Task-agnostic exoskeleton control via biological joint moment estimation.

Publisher: Nature Publishing Group Country of Publication: England NLM ID: 0410462 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1476-4687 (Electronic) Linking ISSN: 00280836 NLM ISO Abbreviation: Nature Subsets: MEDLINE

Publication: Basingstoke : Nature Publishing Group
Original Publication: London, Macmillan Journals ltd.

Biomechanical Phenomena* ; Deep Learning* ; Exoskeleton Device* ; Lower Extremity*/physiology ; Neural Networks, Computer*; Adult ; Female ; Humans ; Male ; Young Adult ; Energy Metabolism/physiology ; Hip Joint/physiology ; Knee Joint/physiology ; Locomotion ; Running/physiology ; Torque ; Walking/physiology ; Weight-Bearing/physiology

Lower-limb exoskeletons have the potential to transform the way we move ^1-14 , but current state-of-the-art controllers cannot accommodate the rich set of possible human behaviours that range from cyclic and predictable to transitory and unstructured. We introduce a task-agnostic controller that assists the user on the basis of instantaneous estimates of lower-limb biological joint moments from a deep neural network. By estimating both hip and knee moments in-the-loop, our approach provided multi-joint, coordinated assistance through our autonomous, clothing-integrated exoskeleton. When deployed during 28 activities, spanning cyclic locomotion to unstructured tasks (for example, passive meandering and high-speed lateral cutting), the network accurately estimated hip and knee moments with an average R ² of 0.83 relative to ground truth. Further, our approach significantly outperformed a best-case task classifier-based method constructed from splines and impedance parameters. When tested on ten activities (including level walking, running, lifting a 25 lb (roughly 11 kg) weight and lunging), our controller significantly reduced user energetics (metabolic cost or lower-limb biological joint work depending on the task) relative to the zero torque condition, ranging from 5.3 to 19.7%, without any manual controller modifications among activities. Thus, this task-agnostic controller can enable exoskeletons to aid users across a broad spectrum of human activities, a necessity for real-world viability.
Competing Interests: Competing interests D.D.M. and A.J.Y. are inventors on a patent filed with the US Patent Office (US 18/340,981) by the Georgia Institute of Technology that covers some of the methods for state estimation described in this work. X, The Moonshot Factory, funded this work, contributed to its conceptualization and methodological design, and contributed much of the exoskeleton hardware and software used in this study.
(© 2024. The Author(s), under exclusive licence to Springer Nature Limited.)

Kim, J. et al. Reducing the metabolic rate of walking and running with a versatile, portable exosuit. Science 365, 668–672 (2019). (PMID: 31416958)
Witte, K. A., Fiers, P., Sheets-Singer, A. L. & Collins, S. H. Improving the energy economy of human running with powered and unpowered ankle exoskeleton assistance. Sci. Robot. 5, eaay9108 (2020). (PMID: 33022600)
Slade, P., Kochenderfer, M. J., Delp, S. L. & Collins, S. H. Personalizing exoskeleton assistance while walking in the real world. Nature 610, 277–282 (2022). (PMID: 362244159556303)
Awad, L. N. et al. A soft robotic exosuit improves walking in patients after stroke. Sci. Transl. Med. 9, eaai9084 (2017). (PMID: 28747517)
Malcolm, P., Derave, W., Galle, S. & Clercq, D. D. A simple exoskeleton that assists plantarflexion can reduce the metabolic cost of human walking. PLoS ONE 8, e56137 (2013). (PMID: 234185243571952)
Mooney, L. M., Rouse, E. J. & Herr, H. M. Autonomous exoskeleton reduces metabolic cost of human walking during load carriage. J. NeuroEngineering Rehabil. 11, 80 (2014).
Ishmael, M. K., Archangeli, D. & Lenzi, T. Powered hip exoskeleton improves walking economy in individuals with above-knee amputation. Nat. Med. 27, 1783–1788 (2021). (PMID: 34635852)
Franks, P. W. et al. Comparing optimized exoskeleton assistance of the hip, knee, and ankle in single and multi-joint configurations. Wearable Technol. 2, e16 (2021). (PMID: 3848663310936256)
Baltrusch, S. J. et al. The effect of a passive trunk exoskeleton on metabolic costs during lifting and walking. Ergonomics 62, 903–916 (2019). (PMID: 30929608)
Lim, B. et al. Delayed output feedback control for gait assistance with a robotic hip exoskeleton. IEEE Trans. Robot. 35, 1055–1062 (2019).
Zhang, J. et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science 356, 1280–1284 (2017). (PMID: 28642437)
Shepherd, M. K., Molinaro, D. D., Sawicki, G. S. & Young, A. J. Deep learning enables exoboot control to augment variable-speed walking. IEEE Robot. Autom. Lett. 7, 3571–3577 (2022).
Gasparri, G. M., Luque, J. & Lerner, Z. F. Proportional joint-moment control for instantaneously adaptive ankle exoskeleton assistance. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 751–759 (2019). (PMID: 30908231)
Molinaro, D. D., Kang, I. & Young, A. J. Estimating human joint moments unifies exoskeleton control, reducing user effort. Sci. Robot. 9, eadi8852 (2024). (PMID: 38507475)
Siviy, C. et al. Opportunities and challenges in the development of exoskeletons for locomotor assistance. Nat. Biomed. Eng. 7, 456–472 (2023). (PMID: 36550303)
Sawicki, G. S., Beck, O. N., Kang, I. & Young, A. J. The exoskeleton expansion: improving walking and running economy. J. NeuroEngineering Rehabil. 17, 25 (2020).
Collins, S. H., Wiggin, M. B. & Sawicki, G. S. Reducing the energy cost of human walking using an unpowered exoskeleton. Nature 522, 212–215 (2015). (PMID: 258308894481882)
Yang, J., Park, J., Kim, J., Park, S. & Lee, G. Reducing the energy cost of running using a lightweight, low-profile elastic exosuit. J. NeuroEngineering Rehabil. 18, 129 (2021).
Li, Y. D. & Hsiao-Wecksler, E. T. Gait mode recognition and control for a portable-powered ankle-foot orthosis. In Proc. 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR) 1–8 (IEEE, 2013).
Laschowski, B., McNally, W., Wong, A. & McPhee, J. Environment classification for robotic leg prostheses and exoskeletons using deep convolutional neural networks. Front. Neurorobotics 15, 730965 (2022).
Kang, I., Molinaro, D. D., Choi, G., Camargo, J. & Young, A. J. Subject-independent continuous locomotion mode classification for robotic hip exoskeleton applications. IEEE Trans. Biomed. Eng. 69, 3234–3242 (2022). (PMID: 35389859)
Camargo, J., Flanagan, W., Csomay-Shanklin, N., Kanwar, B. & Young, A. A machine learning strategy for locomotion classification and parameter estimation using fusion of wearable sensors. IEEE Trans. Biomed. Eng. 68, 1569–1578 (2021). (PMID: 33710951)
Qian, Y. et al. Predictive locomotion mode recognition and accurate gait phase estimation for hip exoskeleton on various terrains. IEEE Robot. Autom. Lett. 7, 6439–6446 (2022).
Medrano, R. L., Thomas, G. C., Keais, C. G., Rouse, E. J. & Gregg, R. D. Real-time gait phase and task estimation for controlling a powered ankle exoskeleton on extremely uneven terrain. IEEE Trans. Robot. 39, 2170–2182 (2023). (PMID: 3730423110249462)
Kang, I. et al. Real-time gait phase estimation for robotic hip exoskeleton control during multimodal locomotion. IEEE Robot. Autom. Lett. 6, 3491–3497 (2021). (PMID: 346168998488948)
Huo, W. et al. Impedance modulation control of a lower-limb exoskeleton to assist sit-to-stand movements. IEEE Trans. Robot. 38, 1230–1249 (2022).
Yang, X. et al. Spine-inspired continuum soft exoskeleton for stoop lifting assistance. IEEE Robot. Autom. Lett. 4, 4547–4554 (2019).
Shepherd, M. K. & Rouse, E. J. Design and validation of a torque-controllable knee exoskeleton for sit-to-stand assistance. IEEE ASME Trans. Mechatron. 22, 1695–1704 (2017).
Orendurff, M. S., Schoen, J. A., Bernatz, G. C., Segal, A. D. & Klute, G. K. How humans walk: bout duration, steps per bout, and rest duration. J. Rehabil. Res. Dev. 45, 1077–1089 (2008). (PMID: 19165696)
Winter, D. in Biomechanics and Motor Control of Human Movement Ch. 5, 107–138 (John Wiley & Sons, Ltd, 2009).
Dorschky, E. et al. CNN-based estimation of sagittal plane walking and running biomechanics from measured and simulated inertial sensor data. Front. Bioeng. Biotechnol. 8, 604 (2020). (PMID: 326710327333079)
Molinaro, D. D., Kang, I., Camargo, J., Gombolay, M. C. & Young, A. J. Subject-independent, biological hip moment estimation during multimodal overground ambulation using deep learning. IEEE Trans. Med. Robot. Bionics 4, 219–229 (2022).
Camargo, J., Molinaro, D. & Young, A. Predicting biological joint moment during multiple ambulation tasks. J. Biomech. 134, 111020 (2022). (PMID: 35228154)
Hossain, M. S. B., Guo, Z. & Choi, H. Estimation of lower extremity joint moments and 3D ground reaction forces using IMU sensors in multiple walking conditions: a deep learning approach. IEEE J. Biomed. Health Inform. https://doi.org/10.1109/JBHI.2023.3262164 (2023).
Lin, J., Divekar, N. V., Thomas, G. C. & Gregg, R. D. Optimally biomimetic passivity-based control of a lower-limb exoskeleton over the primary activities of daily life. IEEE Open J. Control Syst. 1, 15–28 (2022). (PMID: 356736059170045)
Zhang, J., Lin, J., Peddinti, V. & Gregg, R. D. Optimal energy shaping control for a backdrivable hip exoskeleton. In Proc. 2023 American Control Conference (ACC) 2065–2070 (IEEE, 2023).
Fang, Y., Orekhov, G. & Lerner, Z. F. Improving the energy cost of incline walking and stair ascent with ankle exoskeleton assistance in cerebral palsy. IEEE Trans. Biomed. Eng. 69, 2143–2152 (2022). (PMID: 349414959254331)
Bishe, S. S. P. A., Nguyen, T., Fang, Y. & Lerner, Z. F. Adaptive ankle exoskeleton control: validation across diverse walking conditions. IEEE Trans. Med. Robot. Bionics 3, 801–812 (2021).
Tagoe, E. A., Fang, Y., Williams, J. R. & Lerner, Z. F. Walking on real-world terrain with an ankle exoskeleton in cerebral palsy. IEEE Trans. Med. Robot. Bionics 6, 202–212 (2024). (PMID: 39130167)
Young, A. J. & Ferris, D. P. State of the art and future directions for lower limb robotic exoskeletons. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 171–182 (2017). (PMID: 26829794)
Delp, S. L. et al. OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomed. Eng. 54, 1940–1950 (2007). (PMID: 18018689)
Seth, A. et al. OpenSim: simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLoS Comput. Biol. 14, e1006223 (2018). (PMID: 300484446061994)
Scherpereel, K., Molinaro, D., Inan, O., Shepherd, M. & Young, A. A human lower-limb biomechanics and wearable sensors dataset during cyclic and non-cyclic activities. Sci. Data 10, 924 (2023). (PMID: 3812942210740031)
Ding, Y. et al. Effect of timing of hip extension assistance during loaded walking with a soft exosuit. J. NeuroEngineering Rehabil. 13, 87 (2016).
Ingraham, K. A., Tucker, M., Ames, A. D., Rouse, E. J. & Shepherd, M. K. Leveraging user preference in the design and evaluation of lower-limb exoskeletons and prostheses. Curr. Opin. Biomed. Eng. 28, 100487 (2023).
Winter, D. A. Biomechanical motor patterns in normal walking. J. Mot. Behav. 15, 302–330 (1983). (PMID: 15151864)
Browning, R. C., Modica, J. R., Kram, R. & Goswami, A. The effects of adding mass to the legs on the energetics and biomechanics of walking. Med. Sci. Sports Exerc. 39, 515 (2007). (PMID: 17473778)
Farris, D. J. & Sawicki, G. S. The mechanics and energetics of human walking and running: a joint level perspective. J. R. Soc. Interface 9, 110–118 (2012). (PMID: 21613286)
Farris, D. J., Hampton, A., Lewek, M. D. & Sawicki, G. S. Revisiting the mechanics and energetics of walking in individuals with chronic hemiparesis following stroke: from individual limbs to lower limb joints. J. NeuroEngineering Rehabil. 12, 24 (2015).
Farris, D. J. & Sawicki, G. S. Linking the mechanics and energetics of hopping with elastic ankle exoskeletons. J. Appl. Physiol. 113, 1862–1872 (2012). (PMID: 23065760)
Camargo, J., Ramanathan, A., Flanagan, W. & Young, A. A comprehensive, open-source dataset of lower limb biomechanics in multiple conditions of stairs, ramps, and level-ground ambulation and transitions. J. Biomech. 119, 110320 (2021). (PMID: 33677231)
Reznick, E. et al. Lower-limb kinematics and kinetics during continuously varying human locomotion. Sci. Data 8, 282 (2021). (PMID: 347118568553836)
Winter, D. A., Sidwall, H. G. & Hobson, D. A. Measurement and reduction of noise in kinematics of locomotion. J. Biomech. 7, 157–159 (1974). (PMID: 4837552)
Molinaro, D. D., Park, E. O. & Young, A. J. Anticipation and delayed estimation of sagittal plane human hip moments using deep learning and a robotic hip exoskeleton. In Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 12679–12685 (IEEE, 2023).
Bai, S., Kolter, J. Z. & Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. Preprint at https://doi.org/10.48550/arXiv.1803.01271 (2018).
Golovin, D. et al. Google Vizier: a service for black-box optimization. In Proc. 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 1487–1495 (Association for Computing Machinery, 2017).
Lim, H., Kim, B. & Park, S. Prediction of lower limb kinetics and kinematics during walking by a single IMU on the lower back using machine learning. Sensors 20, 130 (2020).
Mundt, M. et al. A comparison of three neural network approaches for estimating joint angles and moments from inertial measurement units. Sensors 21, 4535 (2021). (PMID: 342830808271391)
Ancillao, A., Tedesco, S., Barton, J. & O’Flynn, B. Indirect measurement of ground reaction forces and moments by means of wearable inertial sensors: a systematic review. Sensors 18, 2564 (2018). (PMID: 300816076111315)
Forner-Cordero, A., Koopman, H. J. F. M. & van der Helm, F. C. T. Inverse dynamics calculations during gait with restricted ground reaction force information from pressure insoles. Gait Posture 23, 189–199 (2006). (PMID: 16399515)
Nuckols, R. W. et al. Mechanics of walking and running up and downhill: a joint-level perspective to guide design of lower-limb exoskeletons. PLoS ONE 15, e0231996 (2020). (PMID: 328577747454943)
Alexander, N., Strutzenberger, G., Ameshofer, L. M. & Schwameder, H. Lower limb joint work and joint work contribution during downhill and uphill walking at different inclinations. J. Biomech. 61, 75–80 (2017). (PMID: 28734544)
Lenton, G. K. et al. Lower-limb joint work and power are modulated during load carriage based on load configuration and walking speed. J. Biomech. 83, 174–180 (2019). (PMID: 30527387)
Poggensee, K. L. & Collins, S. H. How adaptation, training, and customization contribute to benefits from exoskeleton assistance. Sci. Robot. 6, eabf1078 (2021). (PMID: 34586837)
Brockway, J. M. Derivation of formulae used to calculate energy expenditure in man. Hum. Nutr. Clin. Nutr. 41, 463–471 (1987). (PMID: 3429265)
Selinger, J. C. & Donelan, J. M. Estimating instantaneous energetic cost during non-steady-state gait. J. Appl. Physiol. 117, 1406–1415 (2014). (PMID: 25257873)

Date Created: 20241113 Date Completed: 20241113 Latest Revision: 20241114

20241115

10.1038/s41586-024-08157-7

39537888