…

Background

In previous blog, we solved the CartPole-V1 task which tries to balance the pole upright by applying force in the left and right direction on the cart. Although this task is successfully solved with DQN and PPO, we can't control where the cart should be when the pole is balanced. In this blog we try to solve this goal-guided CartPole task.

…

Background

A central pattern generator (CPG) is a biological neural circuit that produces rhythmic motor patterns, such as walking, breathing, flying, or swimming. CPGs are found in humans and many kinds of animals.

For control tasks such as locomotion of bipedal or quadrupedal robots, the action space is both high-dimensional and continuous, finding methods to directly control all joints is very difficult. It's natural to think controlling each leg motion rather than each joint torque may reduce the complexity of the solution.

CPG assumes each leg's motion follows some rhythmic or quasi-periodic pattern, and divides one full-joints control task into 2(biped)/4(quadruped) leg-joints control tasks, this strategy simplifies the problem and also improves interpretability/controllability.

…

…