Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer

•A new variant of attention mechanisms focuses on modeling cross-sentence attention.•A novel model integrates highway attention in Transformer for modeling dialogues.•Our model is capable of modeling complex dialogue-level information.•The results on two response selection datasets show consistent performance.