Neural Networks

Artificial neural networks (ANNs) are state-of-the-art machine learning architectures modeling neurons and their connections through weights and biases. ANNs serve as universal function approximators, meaning that a sufficiently complex neural network can learn almost any function in any dimension. This flexibility, combined with backpropagation and a learning algorithm, enables to learn unknown functions with an astonishing accuracy. This chapter introduces the basics of neural networks, backpropagation, and the learning algorithm. Additionally, activation functions and regularization are treated. The derivatives with respect to the networks’ input are also explained, as these are essential for the upcoming chapters on physics-informed neural networks and the deep energy method. Finally, an outlook on more advanced network architectures is provided.