SpanGNN: Towards Memory-Efficient Graph Neural Networks Via Spanning Subgraph Training

Graph Neural Networks (GNNs) have superior capability in learning graph data.Full-graph GNN training generally has high accuracy, however, it suffers fromlarge peak memory usage and encounters the Out-of-Memory problem when handlinglarge graphs. To address this memory problem, a popular solution is mini-batchGNN training. However, mini-batch GNN training increases the training varianceand sacrifices the model accuracy. In this paper, we propose a newmemory-efficient GNN training method using spanning subgraph, called SpanGNN.SpanGNN trains GNN models over a sequence of spanning subgraphs, which areconstructed from empty structure. To overcome the excessive peak memoryconsumption problem, SpanGNN selects a set of edges from the original graph toincrementally update the spanning subgraph between every epoch. To ensure themodel accuracy, we introduce two types of edge sampling strategies (i.e.,variance-reduced and noise-reduced), and help SpanGNN select high-quality edgesfor the GNN learning. We conduct experiments with SpanGNN on widely useddatasets, demonstrating SpanGNN's advantages in the model performance and lowpeak memory usage.