A deep learning approach for stem base detection of row crop plants

Abstract

This work introduces two deep learning-based methods: DeepRow for the detection of crop rows and DeepStem for identifying the stem bases of individual plants within crop rows. A central focus of this research lies in the development of training datasets for the DeepRow and DeepStem networks. Specifically, we investigate whether the performance of these networks can be enhanced by supplementing training datasets consisting of real field images with synthetically generated images. Various variants of farmland image datasets were created, including those consisting of 100% real images, 100% synthetic images,and mixed datasets. Real images were manually annotated using the Label Image annotation tool, while a modification tool from the computer game Farming Simulator 22 was employed for the automated generation and annotation of synthetic images. This study aims to examine the effectiveness of synthetic farmland images in improving the performance of the DeepRow and DeepStem models. Additionally, a part of the study explores the effects of different virtual plant and ground textures in the generation of synthetic images.Regarding DeepRow, the results demonstrate that augmenting real datasets with synthetic images positively influences performance. Furthermore, the chosen approach of texture editing contributes positively to crop row detection accuracy. Specifically, a dataset consisting of 20% real images and 80% texture-edited synthetic images yielded a 9.06% improvement in accuracy and a 6.4% improvement in IoU compared to a dataset consisting solely of real farmland images. In contrast, the performance of DeepStem models exhibited a negative trend when synthetic images were added to the training set. Additionally, the proposed approach of texture editing tended to reduce performance metrics for DeepStem models.