Towards an End-to-End Framework for Invasive Brain Signal Decoding with Large Language Models
Interspeech 2024(2024)
摘要
In this paper, we introduce a groundbreaking end-to-end (E2E) framework fordecoding invasive brain signals, marking a significant advancement in the fieldof speech neuroprosthesis. Our methodology leverages the comprehensivereasoning abilities of large language models (LLMs) to facilitate directdecoding. By fully integrating LLMs, we achieve results comparable to thestate-of-the-art cascade models. Our findings underscore the immense potentialof E2E frameworks in speech neuroprosthesis, particularly as the technologybehind brain-computer interfaces (BCIs) and the availability of relevantdatasets continue to evolve. This work not only showcases the efficacy ofcombining LLMs with E2E decoding for enhancing speech neuroprosthesis but alsosets a new direction for future research in BCI applications, underscoring theimpact of LLMs in decoding complex neural signals for communicationrestoration. Code will be made available athttps://github.com/FsFrancis15/BrainLLM.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要