TMRL-NBV: Triangular mesh-based reinforcement learning for next-best-view in active 3D reconstruction

¹ Institute of Artificial Intelligence, Beihang University, Beijing 100191, China
² School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China

Abstract

Next-best-view (NBV) planning is essential in active 3D reconstruction, aiming to select informative viewpoints to improve coverage and efficiency. This work proposes Triangular mesh-based reinforcement learning for NBV (TMRL-NBV), formulating NBV selection as a Markov Decision Process. The framework integrates a structured observation space, a continuous spherical action space, and a field-of-view constrained raycasting mechanism for triangle-level visibility estimation. A composite reward function encourages surface coverage, viewpoint novelty, and trajectory efficiency. The policy is optimized using Proximal Policy Optimization. Experiments on the Mechanical Components Benchmark test split and real mechanical part meshes collected from external sources demonstrate that TMRL-NBV achieves high reconstruction completeness, validating its effectiveness in general 3D vision tasks.