Material-integrated Prediction, Control, and Distributed Learning in Soft Robots

Aguasvivas Manzano, Sarah

Graduate Thesis Or Dissertation

Material-integrated Prediction, Control, and Distributed Learning in Soft Robots Public Deposited

Analytics

Download PDF

Citations

Citeable URL: https://scholar.colorado.edu/concern/graduate_thesis_or_dissertations/d217qr00s

Abstract

Soft roboticists promise lofty goals around soft robots, such as Mars, deep-sea, and internal human body exploration. Yet, roboticists design and build algorithms that make the robot reliant on personal computers or clouds for important functions. In most soft robotic designs, a low-power microcontroller unit (MCU) collects sensor signals and sends actuation commands from the material. This work focuses on making the first steps towards computationally untethering advanced prediction, control, and learning algorithms onto the already existing MCUs in the soft robot material by considering the algorithm’s memory needs and power consumption. The first step towards this work is the proposal, building, deployment, and open-sourcing of a compiler (nn4mc, https://nn4mc.com) that allows researchers and hobbyists to translate neural network inference trained in Tensorflow into C code that is interpretable by generalized low-power MCUs. The second step consists of untethering real-time high-bandwidth nonlinear control that uses neural network forward kinematic models compiled using nn4mc and real-time optimization toward nonlinear predictive control that consumes less than 46mW of average power and controls soft actuators with less than 2 mm of error for a path following task. The third step consists of encoding sensor signals from the soft robot onto a space where active learning becomes a trivial task through observation embeddings and graph-based distributed semi-supervised learning without the need for backpropagation at the low power memory-restricted compute units in a simulated study. nn4mc has sparked collaborations with many researchers, such as real-time prediction in prosthetic fingertips, aeroelastic-aware airplane wings, and real-time pose classification on a wearable garment worn by drag queens. Taken together, this body of work presents tools and methods to deliver on the promise of computationally untethered soft robots; that is, accessible firmware engineering tractable at low-power compute, which belongs in the future of soft robotics.

Creator