My research endeavors revolve around the generative pre-training paradigm of NLP, with a particular emphasis on:
General LLM pre-training, prompting, instruction tuning, and their variants across tasks and modalities;
Augmented LLMs with non-parametric priors;
LLM alignment with human preferences.