My research interest generally lies in scalability and the interpretability aspects of Natural Language Processing, with their applications to Language Modeling, Information Extraction and Dialogue System. I am striving to answer the following research questions:

Can deep neural networks develop its own language (i.e., sparse, structured and discrete latent representations) during large-scale self-supervised learning?
Can we leverage sparse latent decisions to improve generalizability, interpretability and scalability of the foundation models?