The Nelder–Mead simplex algorithm with perturbed centroid for high-dimensional function optimization

It is a well known fact that the widely used Nelder–Mead (NM) optimization method soon becomes inefficient as the dimension of the problem grows. It has been observed that part of the reason for the inefficiency is that the search direction becomes practically perpendicular to the direction of the local downhill gradient. In this paper, we identify which operations are responsible for the increase of the angle between the search direction and the direction of the local downhill gradient. We show that it is possible to decrease this angle by randomly perturbing the centroid of the n best vertices around which the original NM algorithm moves the worst vertex. Using a perturbed centroid, the algorithm outperforms the standard NM method as well as the NM method with adaptive parameters for problems with higher dimensions, and works well for problems with dimensions of well over 100.