arXiv cs.LG + cs.RO + cs.HC· Leyuan Jin, Jiahao Mei, Yixuan Yuan, Shimeng Li, Mengyue Wu, Yue Ding·· 1 天前AI 评分42
EEG-Captioner：以信号基本属性为锚的 EEG 到语言生成
EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties
AI 导读
研究团队构建了含 160,781 个片段的 EEG-语言数据集，每段配有信号级问答对和基于特征的 EEG 描述，并提出联合建模时域波形与频谱-空间表示的双域 EEG 编码器。基于该编码器的 EEG-Captioner 在信号级问答上达到 88.81% 的 Macro-BA，描述事实一致性从 82.06% 提升至 83.46%。
正文 · 原文
Abstract:Electroencephalography (EEG)-to-language models offer a promising approach for translating complex neural signals into human-readable descriptions. However, generating plausible language does not necessarily demonstrate that a model has learned to interpret the underlying EEG signal. A direct way to assess such interpretation is to probe objectively measurable properties of the signal itself. We therefore construct an EEG-language dataset of 160,781 segments annotated with signal-level question answering (QA) pairs, where each question targets one of five property families and requires a quantitative comparison involving spectral, spatial, temporal, or lateral signal properties. Each segment is also paired with a feature-grounded EEG caption. We further propose a Dual-Domain EEG Encoder that jointly models temporal waveforms and spectral-spatial representations, and introduce EEG-Captioner, a unified EEG-to-language framework trained with both signal-level QA and EEG caption supervision. Experiments show that the proposed encoder achieves the highest overall macro-averaged balanced accuracy (Macro-BA) among the compared EEG encoders, while EEG-Captioner achieves a Macro-BA of 88.81% on signal-level QA and improves caption factual consistency from 82.06% to 83.46% over caption-only language adaptation, demonstrating the complementary benefits of signal-level QA and caption supervision.
| Comments: | 5 pages, 2 figures, 3 tables. Submitted to ICASSP 2027 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.30995 [cs.LG] |
| (or arXiv:2610.30995v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30995 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leyuan Jin [view email]
[v1]
Thu, 22 Oct 2026 09:37:53 UTC (395 KB)
来源：arXiv cs.LG + cs.RO + cs.HC · arxiv.org