Exploring Task Unification in Graph Representation Learning via Generative Approach
CoRR(2024)
Abstract
Graphs are ubiquitous in real-world scenarios and encompass a diverse range
of tasks, from node-, edge-, and graph-level tasks to transfer learning.
However, designing specific tasks for each type of graph data is often costly
and lacks generalizability. Recent endeavors under the "Pre-training +
Fine-tuning" or "Pre-training + Prompt" paradigms aim to design a unified
framework capable of generalizing across multiple graph tasks. Among these,
graph autoencoders (GAEs), generative self-supervised models, have demonstrated
their potential in effectively addressing various graph tasks. Nevertheless,
these methods typically employ multi-stage training and require adaptive
designs, which on one hand make it difficult to be seamlessly applied to
diverse graph tasks and on the other hand overlook the negative impact caused
by discrepancies in task objectives between the different stages. To address
these challenges, we propose GA^2E, a unified adversarially masked autoencoder
capable of addressing the above challenges seamlessly. Specifically, GA^2E
proposes to use the subgraph as the meta-structure, which remains consistent
across all graph tasks (ranging from node-, edge-, and graph-level to transfer
learning) and all stages (both during training and inference). Further, GA^2E
operates in a "Generate then Discriminate" manner. It leverages the
masked GAE to reconstruct the input subgraph whilst treating it as a generator
to compel the reconstructed graphs resemble the input subgraph. Furthermore,
GA^2E introduces an auxiliary discriminator to discern the authenticity between
the reconstructed (generated) subgraph and the input subgraph, thus ensuring
the robustness of the graph representation through adversarial training
mechanisms. We validate GA^2E's capabilities through extensive experiments on
21 datasets across four types of graph tasks.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined