Intelligent vision for the detection of chemistry glassware toward AI robotic chemists

One of the key steps to make an artificially intelligent (AI) and robotic chemist is the introduction of machine vision for guiding the experiment operation in the AI-redefined laboratory. In order to realize the targets, the prerequisites are to innovate/implement the intelligent vision for the detection of chemistry glassware. Here, we reported a computer vision method based on You only look once (YOLO) with a self-developed Chemical Vessel Identification Dataset (CViG) for the improvement of classification and recognition performance. The training dataset has been collected that includes 4072 images in real-time chemical laboratory. Three models, YOLOv5s, Slim-YOLOv5s and YOLOv7, have been exploited for the recognition of seven types of glassware in the condition of different scenarios (recognition distance, light and dark, stationary and moving). The improved Slim-YOLOv5s exhibited better recognition ability in various scenes, and the recognition accuracy of chemical vessels is improved by 1.51 % compared with YOLOv5s, and the size of the model is reduced from 14.4 MB to 11.0 MB. Slim-YOLOv5s's mAP is similar to YOLOv7's ability with a disadvantage of large volume, suggested that the improved Slim-YOLOv5s clearly has more advantages in terms of embedded requirements. This vision-assisted system capable of classifying chemical containers accurately in the scenarios of real-time chemical experiments will provide a good vision solution in the frontier fields of automated machine chemistry.