Towards an Interactive Drone, a Reinforcement Learning Approach


Asem Khattab
Presentation MSc presentation
Date 2018-08-31
Time 15:00
Location Carré 2G

For a drone, equipped with an impedance controller, to correctly deal with physical contact and interaction with the environment, the impedance parameters should be adjusted correctly depending on the interaction task. Concentrating on interaction tasks requiring a constant set of impedance parameters values throughout operation, a model-free learning framework is proposed to automatically find the suitable parameters values. The framework relies on Bayesian optimization and episodic reward calculation requiring the drone to repeatedly perform a predetermined task in the environment actively searching in the impedance parameters space.

The sample-efficiency and safety of learning were improved by adding two novel modifications to Bayesian optimization. The first one is local optimization of acquisition functions allowing learning to get early warnings during exploration. The second one is conditioning the reward signal with a logistic function exploiting the initial knowledge and enabling generalizing learning settings to different situations.

The proposed technique was validated by applying it to learn the suitable impedance parameters values for a simulated drone performing sliding tasks. The results show that the proposed framework is able to automatically find suitable impedance parameters values in different situations given the same initial knowledge and that the learned parameters values can be generalized to similar interaction tasks.

Posted on Friday, July 6, 2018