I propose a new contrastive loss function, namely the Generalized Contrastive Loss, that relies on a graded definition of image similarity ground truth. This function, together with the new ground truth allows us to train a siamese architecture without relying on overcomplicated pair mining strategies and using only simple pooling layers (GeM and average), outperforming more complex state-of-the-art approaches like NetVLAD.