20.5 C-Transformer: A 2.6-18.1Μj/token Homogeneous DNN-Transformer/Spiking-Transformer Processor with Big-Little Network and Implicit Weight Generation for Large Language Models

Recently, transformer-based large language models (LLMs), shown in Fig. 20.5.1, are widely used, and even on-device LLM systems with real-time responses are anticipated [1]. Many transformer processors [2–4] enhance energy efficiency by increasing hardware utilization and reducing power consumption, but their system power consumption and response time are still not suitable for mobile devices. Since LLMs, such as GPT-2, have many parameters (400-700M), their External Memory Access (EMA) consumes 68% of the total power. Pruning can increase sparsity to alleviate the EMA bottleneck, but can be applied only to a simple task (language modeling) to predict the next word [5]. For high-level tasks such as translation, question answering, and summarization, it is impossible to achieve high sparsity (<~40%) [6].