Analyzing the interplay between transferable GANs and gradient optimizers

Recently, different neuroevolutionary approaches have been proposed to enhance the architecture and hyperparameters of generative neural models. Many of these approaches have to train the candidate architectures before their evaluation. The gradient descent algorithm used to do so is often overlooked in the evolutionary procedure, despite being a critical aspect of the training. In this paper, we investigate the role played by the gradient-based optimizer chosen to train a generative adversarial network when its architecture has been obtained using neuroevolution. We focus on 2D Gaussian mixture approximation problem and evaluate the effect of a set of representative gradient-based techniques on the quality of the performance of the GANs. Our results show that the particular choice of the gradient optimizer can be as relevant as the appropriate selection of the architecture.