Explainable and data-efficient learning for visual guidance of autonomous agri-robots

Abstract

To feed a growing world population and achieve the goal of zero hunger, we must develop new technologies to improve farm productivity and sustainability. Agri-robots can be a part of this solution, but new research is needed to provide reliable and low-cost autonomous operation across the broad spectrum of agricultural environments. Combining low-cost RGB cameras for vision with the recent advances in deep learning is a promising direction that can enable easier adaption and lower hardware costs than existing solutions.

We explicitly tackle two of the main challenges faced when applying deep learning in robotics: learning from data of limited quantity and/or quality, and making neural networks easier to understand for humans. Thus, the main objectives of this work are to develop and apply methods that are more data-efficient and explainable than state-of-the-art in learning-based visual robot guidance, and to apply this insight to guide agri-robots in the field.

These topics are explored through five papers. First, we investigate the properties of an established end-to-end learning strategy for guidance and apply it in crop row following. Although promising at first, the black-box nature of this approach and inherent difficulties for debugging led to two different strategies; 1) a more explainable network architecture with a new supervision strategy for this task, and 2) a novel visualisation method to better understand visual features in convolutional neural networks. Finally, we unite these strategies in a new hybrid learning approach for row following that is both robust, data-efficient and more transparent.

The main contributions of this thesis are 1) Increased explainability through the development of a novel feature visualisation method, which provides explanations that are complementary to existing methods, 2) Increased data-efficiency and adaptability of learning-based crop row following through a new supervision approach which eliminates the need for hand-drawn labels, and 3) New insight into applications of learning-based methods in the field, by testing several supervision strategies on a real robot in the field, and considering the whole pipeline from data collection to predicted steering angle.