Enhancing indoor place classification for mobile robots using RGB-D data and deep learning architectures

¹ Centre for Robotics and Future Production, Future Production: Manufacturing, CSIR, South Africa
² Industry Connect, Future Production: Manufacturing, CSIR, South Africa

^* Corresponding author: bveden@csir.co.za

Abstract

Place classification is crucial for a robot's ability to make high- level decisions. When a robot can identify its operating environment, it can provide more appropriate services. This capability is similar to how humans use their understanding of their surroundings to make informed decisions about appropriate actions. Depth data offers valuable spatial information that can enhance place classification on a robot. However, it is more common for mobile robot applications to rely on RGB data rather than RGB-D data for classifying indoor places. This study demonstrates that incorporating depth information improves the classification of indoor places using a mobile robot. Data were collected from a mobile robot, and indoor scenes were classified based on RGB and RGB-D inputs. A comparison was made between the performance of VGG16, Inception v3, and ResNet50 architectures using RGB data alone. Subsequently, depth information was fused with these RGB models. Experiments showed that classification accuracy improved when tested on the mobile robot by including depth data. In the experiment, the robot created a map of the indoor environment and identified four different rooms on the map using the trained models. This demonstrates the enhanced classification capabilities achieved by incorporating depth information.