Self-supervised imitation learning from observation

• Ano de defesa: 2021
• Autor(a) principal: Gavenski, Nathan Schneider
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Pontifícia Universidade Católica do Rio Grande do Sul; Escola Politécnica; Brasil; PUCRS; Programa de Pós-Graduação em Ciência da Computação
• Palavras-chave em Português: Imitation Learning; Behavioral Cloning; Self-supervised Learning; Aprendizado por Imitação; Clonagem de Comportamento; Aprendizado Auto-Supervisionado; CIENCIA DA COMPUTACAO::TEORIA DA COMPUTACAO
• Link de acesso: http://tede2.pucrs.br/tede2/handle/tede/9778

Humans have the ability to learn through observation. The computational equivalent of learning by observation is behavioral cloning, an imitation learning technique that teaches an agent how to behave through expert demonstrations. Recent approaches work towards making use of unlabeled data with fully-observable snapshots of the states, decoding the observed information into actions in a self-supervised fashion. However, there are several problems still left to be addressed, including the many times the iterative learning scheme gets stuck into bad local minima. In this work, we propose three different methods, Augmented Behavioral Cloning from Observation, Imitating Unknown Policies via Exploration, and Combined Reinforcement and Imitation Learning, which aim to solve the problems of the decaying learning process, nonexplorative policies, and sample efficiency during the iterative process. The results from Augmented Behavioral Cloning from Observations show that a sampling mechanism can create more appropriate iterative learning cycles, while Imitating Unknown Policies via Exploration results convey that an exploration strategy can achieve results even better than the expert, reaching the state-of-the-art of the task. Lastly, the Combined Reinforcement and Imitation Learning framework shows that adding a reinforcement learning method within the imitation learning framework can create more efficient policies and reach similar results to the second method with fewer samples. Both the second and the third methods offer distinct trade-offs between performance and efficiency, depending on the difficulty of acquiring expert samples.