Peek-a-bot: learning through vision in Unreal Engine
Loading...
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Humans learn to navigate and interact with their surroundings through their senses, particularly vision. Ego-vision has lately become a significant focus in computer vision, enabling neural networks to learn from first-person data effectively, as we humans do. Supervised or self-supervised learning of depth, object location and segmentation maps through deep networks has shown considerable success in recent years. On the other hand, reinforcement learning (RL) has been focusing on learning from different kinds of sensing data, such as rays, collisions, distances, and other types of observations. In this paper, we merge the two approaches, providing a complete pipeline to train reinforcement learning agents inside virtual environments, only relying on vision, eliminating the need for traditional RL observations. We demonstrate that visual stimuli, if encoded by a carefully designed vision encoder, can provide informative observations, thus replacing ray-based approaches and drastically simplifying the reward shaping typical of classical RL. Our method is fully implemented inside Unreal Engine 5, from the realtime inference of visual features to the online training of the agents' behaviour using the Proximal Policy Optimization (PPO) algorithm. To the best of our knowledge, this is the first in-engine solution targeting video games and simulation, enabling game developers to easily train vision-based RL agents without writing a single line of code. All the code, complete experiments and analysis will be available at https://mmlab-cv.github.io/Peek-a-bot/.
Description
@inproceedings{10.2312:stag.20241330,
booktitle = {Smart Tools and Applications in Graphics - Eurographics Italian Chapter Conference},
editor = {Caputo, Ariel and Garro, Valeria and Giachetti, Andrea and Castellani, Umberto and Dulecha, Tinsae Gebrechristos},
title = {{Peek-a-bot: learning through vision in Unreal Engine}},
author = {Pietra, Daniele Della and Garau, Nicola and Conci, Nicola and Granelli, Fabrizio},
year = {2024},
publisher = {The Eurographics Association},
ISSN = {2617-4855},
ISBN = {978-3-03868-265-3},
DOI = {10.2312/stag.20241330}
}