Bandit neural architecture search based on performance evaluation for operation selection

Neural architecture search (NAS) plays an important role in many computer vision tasks. However, the high computational cost of forward and backward propagations during the search, process restricts its practical application. In this paper, we present the search process as a multi-armed bandit problem, where we take into account the uncertainty caused by the contradiction between the huge search space and limited number of trials. Bandit NAS optimizes the trade-off between exploitation and exploration for a highly efficient search. Specifically, we sampled from a set of operations in one trial, where each operation was weighted by its trial performance and a bias to allow operations with less training to be selected. We further reduced the search space by abandoning the operation with the lowest potential, significantly reducing the search cost. Experimental results on the CIFAR-10 dataset show that the resulting architecture achieves the most advanced precision with a search speed approximately two times faster than that of partially connected differentialble architecture search. On ImageNet, we attained the most advanced top-1 accuracy of 75.3% with a search time of 1.8 GPU days.