Dynamic Programming of a Torso Actuated Rimless Wheel Robot

This thesis presents a deliberation in the application of Dynamic Programming for the variable speed control of a simplified model of the Rowdy Runner II, a torso actuated rimless wheel robot. Rowdy consists of a torso sandwiched between two rimless wheels. By applying a torque between the torso and wheels, forward motion is produced when the torso's center of mass protrudes beyond the contact point of the leading foot with respect to the gravity vector. Currently, the Rowdy Runner II is capable of controlling the angle of the torso during motion to maintain a constant velocity. The work presented in this thesis uses Dynamic Programming for optimizing the energy usage for achieving the speed transition. One issue, however, is that the accuracy of the Dynamic Programming solution depends on the resolution of the grid. In practice, Dynamic Programming is known to require disproportionally larger amounts of computational time as the complexity of a problem grows, a relation known as the Curse of Dimensionality. This creates a dilemma: on the one hand, a fine grid is essential for good controller performance while on the other, finding these optimal policies at higher resolution require a rather large amount of time. To circumvent this issue, a step-to-step map, also known as the Poincare map, is used to discretize system's state. The Poincare map relates the mid-stance velocity of the rimless wheel (the state) and the fixed torso angle per step (the control variable) to the mid-stance velocity at the next step (the new state). Using this map, a Dynamic Programming problem to minimize a weighted sum of the square of the deviation from the desired speed and the actuator effort is set up. This approach reduces the number of states to consider, simplifying the Dynamic Programming problem and reducing computational time. The optimal policies for speed transitions are then found using a combination of policy- and value- iteration for computational efficiency. The results of this work show that it is possible to switch from a mid-stance velocity of 2 rad/s to 3 rad/s in less than 6 steps.