A Novel Deep Multi-Image Object Detection Approach for Detecting Alien Barleys in Oat Fields Using RGB UAV Images

Oat products are significant parts of a healthy diet. Pure oat is gluten-free, which makes it an excellent choice for people with celiac disease. Elimination of alien cereals is important not only in gluten-free oat production but also in seed production. Detecting gluten-rich crops such as wheat, rye, and barley in an oat production field is an important initial processing step in gluten-free food industries; however, this particular step can be extremely time consuming. This article demonstrates the potential of emerging drone techniques for identifying alien barleys in an oat stand. The primary aim of this study was to develop and assess a novel machine-learning approach that automatically detects and localizes barley plants by employing drone images. An Unbiased Teacher v2 semi-supervised object-detection deep convolutional neural network (CNN) was employed to detect barley ears in drone images with a 1.5 mm ground sample distance. The outputs of the object detector were transformed into ground coordinates by employing a photogrammetric technique. The ground coordinates were analyzed with the kernel density estimate (KDE) clustering approach to form a probabilistic map of the ground locations of barley plants. The detector was trained using a dataset from a reference data production site (located in Ilmajoki, Finland) and tested using a 10% independent test data sample from the same site and a completely unseen dataset from a commercial gluten-free oats production field in Seinajoki, Finland. In the reference data production dataset, 82.9% of the alien barley plants were successfully detected; in the independent farm test dataset, 60.5% of the ground-truth barley plants were correctly recognized. Our results establish the usefulness and importance of the proposed drone-based ultra-high-resolution red-green-blue (RGB) imaging approach for modern grain production industries.