arXiv cs.LG + cs.RO + cs.HC· Ingo Marquardt, Anthilia Alchanat, Priyanka Jain·· 3 天前AI 评分55
研究：Llama-3.1-8B 深层暴露默读 EEG 的篇章级语境
Deep LLM layers expose passage-level context in EEG during silent reading
AI 导读
研究团队为 Llama-3.1-8B 的每一层各训练一个解码器，用 19 通道 EEG 在默读时解码词汇信息，发现词级可解码性在浅层达到峰值并随层数加深持续下降，而语境追踪成分随深度上升。作者用目标几何对照和跨运行预测置换对照排除了词解码副产物与会话特征的影响，认为单个 0.5 秒 EEG 窗口即可携带可被语境化目标吸收的篇章级信息。
正文 · 原文
Abstract:In a previous report we showed that a decoder can recover open-vocabulary word-level information from 19-channel EEG recorded during silent reading, by aligning short EEG windows with hidden-state embeddings of the presented word taken from a large language model (LLM). That analysis contrasted two of the model's 33 layers: the context-free input embedding and one mid-depth layer. Here we ask how EEG-decodable structure is distributed over the full depth of the language model. We trained one decoder per layer of Llama-3.1-8B, with and without a causal transformer over the sequence of EEG windows. We distinguish word-level decodability from a context-tracking component. Word-level decodability peaks in early layers and declines steadily towards the last layer. In contrast, the context-tracking component of the retrieval gain rises with depth, both with and without a causal transformer. Unexpectedly, a decoder that sees a single 0.5-s window at a time tracks the text passage too. Two new controls dissect this component. A target-geometry control, a synthetic predictor made of the true target plus noise calibrated to each decoder's word-level accuracy, shows that context tracking is not a by-product of word decoding and target geometry. A control that permutes predictions across runs from either the same or from different sessions shows that most of the observed context-tracking is not caused by an EEG recording session signature. Instead, context tracking mostly follows the text. A single EEG window therefore carries passage-level information that a contextual target can absorb. The results locate word identity in the early layers of the language model, discourse state in the deeper layers, and make the choice of target layer a design dial for EEG language decoders.
| Comments: | 41 pages (including 23 pages of Supplementary Material), 12 figures (including 5 in Supplementary Material) |
| Subjects: | Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2610.23638 [cs.LG] |
| (or arXiv:2610.23638v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ingo Marquardt [view email]
[v1]
Fri, 16 Oct 2026 15:41:19 UTC (4,442 KB)
来源：arXiv cs.LG + cs.RO + cs.HC · arxiv.org