A representation-learning game for classes of prediction tasks
ICLR 2024(2024)
摘要
We propose a game-based formulation for learning dimensionality-reducing
representations of feature vectors, when only a prior knowledge on future
prediction tasks is available. In this game, the first player chooses a
representation, and then the second player adversarially chooses a prediction
task from a given class, representing the prior knowledge. The first player
aims is to minimize, and the second player to maximize, the regret: The minimal
prediction loss using the representation, compared to the same loss using the
original features. For the canonical setting in which the representation, the
response to predict and the predictors are all linear functions, and under the
mean squared error loss function, we derive the theoretically optimal
representation in pure strategies, which shows the effectiveness of the prior
knowledge, and the optimal regret in mixed strategies, which shows the
usefulness of randomizing the representation. For general representations and
loss functions, we propose an efficient algorithm to optimize a randomized
representation. The algorithm only requires the gradients of the loss function,
and is based on incrementally adding a representation rule to a mixture of such
rules.
更多查看译文
关键词
representation learning,semi-supervised learning,dimensionality-reduction,regret,minimax solution,mixed strategies,multiplicative weights update
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要