As seen in a previous post, Bayes filters such as Kalman filters can be used to estimate the state of a robot. Usually, this requires having a model of how the robot’s sensor measurements relate to the state you want to observe and a model to predict how the robot’s control inputs impact its state. When little is know about these models, machine learning techniques can help find models automatically.

In the scenario imagined by Ko et al. the goal is to infer a car’s position (state) on a race track based on remote control commands and measurements from an Inertial Measurement Unit (IMU, see red box in picture below) that provides turn rates in roll, pitch, and yaw and accelerations in 3 dimensions. The car is only controlled in speed since a “rail” in the track makes it turn. As a start, Ko et al. collect data by driving the car around the track while recording its remote control commands, IMU measurements, and position (ground truth) estimated using an overhead camera. The data is then used to train probabilistic models (Gaussian Process models) that are finally integrated into a Bayes filter.

However, the need for ground truth requires extra effort and additional hardware such as the overhead camera. To overcome this problem, Ko et al. extend their method to deal with situations where no, little, or noisy ground truth is used to train the models.

Results show the successful tracking of the car with better performance than state-of-the-art approaches, even when no ground truth is given. The authors also show how the developed method can be used to allow the car or a robot arm to replay trajectories based on expert demonstrations.

Related posts

• Teaching motion primitives
• From iCub to artist
• Yeti robot avoids snow traps
• Learning behavioral models
• Social learning
• Learning tasks across different environments
• Classifying dynamic obstacles

It is often difficult to predict the high-level behavior of a robot given low-level models about sensors, actuators and controllers. You might know your robot will turn in response to obstacles but not how it will behave in a room full of people.

Modeling the global behavior of a robot is useful in order to predict how the robot behaves in different environments. Furthermore, once a good model is inferred, it can be used to improve the robot’s controller parameters online.

To model robot behaviors, Infantes et al. use a probabilistic representation called Dynamic Bayesian Networks. The approach is tested using the Rackham RWI B21R museum guide robot shown below that needs to navigate in an open environment with people. The network captures information concerning the robot’s parameters, environment variables, robot state variables and mission variables. The model is then used to optimize the robot behavior for a given environment. During the learning process, robots are rewarded for good behaviors that avoid failures, go fast and are “human-friendly”. Using this approach, the robot fails less, is faster and has better human acceptance than a robot with hand-tuned parameters.

In the future, Infantes et al. plan to use this approach to learn other robotic tasks such as grasping or interacting with humans.

Related posts

• Teaching motion primitives
• From iCub to artist
• Yeti robot avoids snow traps
• Learning Bayesian filters without precise ground truth
• Social learning
• Learning tasks across different environments
• Classifying dynamic obstacles

In the future, robots will be expected to learn a task and execute it in a variety of realistic situations. Reinforcement-learning and planning algorithms are exactly intended for that purpose. However, one of the main challenges is to make sure actions learned in one environment can be used in new and unforeseen situations in real time.

To address this challenge, Stolle et al. have imagined a series of algorithms which they demonstrate on complex tasks such as solving a marble maze or making Boston Dynamic’s Little Dog navigate over complex terrain (see video below).

The first ingredient of success relies on making robots learn what action to take based on local features, meaning features as viewed by the robot (e.g. “there is a wall to the right”). These local features can then be recognized in new environments when the robot is in similar situations. Instead, many existing algorithms use global information, for example by saying “perform this action in position (x,y,z)”. Changing the environment however would typically make these global policies useless.

The second ingredient makes robots build libraries containing sequences of actions (trajectories) that can bring a robot from its current state to an aimed goal. Robots then apply the actions from the trajectory nearest to their state to achieve a task. This strategy is interesting because it is not computationally expensive and does not require large amounts of fast memory.

Finally, don’t miss the following video of little-dog climbing over a fence. This special purpose behavior can be used in a variety of situations.

Related posts

• Teaching motion primitives
• From iCub to artist
• Yeti robot avoids snow traps
• Learning Bayesian filters without precise ground truth
• Learning behavioral models
• Social learning
• Classifying dynamic obstacles