arXiv cs.LG + cs.RO + cs.HC· Kieren Yu, Chang Huang, Ziyang Liu, Kaishun Wu·· 16 小时前AI 评分55
ProtoEEG 用共享视觉原型对齐 EEG 与图像，THINGS-EEG2 检索 Top-1 达 64.0%
ProtoEEG: Structured Prototype Alignment for EEG Visual Decoding
AI 导读
研究团队提出 ProtoEEG，把 EEG 与图像的对齐改为在共享可学习视觉码本上的软组合，EEG 编码器与冻结视觉教师对同一组 K=64 个原型输出分配分布。
正文 · 原文
Abstract:EEG visual decoding remains difficult because scalp recordings are noisy, spatially coarse, and variable across trials and subjects. Most systems align each trial with a dense image embedding, although that embedding mixes semantic content with instance-specific texture, pose, and background. This target can demand details absent from EEG and provides no explicit account of the visual factors supporting a prediction. We propose \textbf{ProtoEEG}, a prototype-mediated framework that reformulates EEG--image alignment as soft composition over a shared learnable visual codebook. A trainable EEG encoder and a frozen visual teacher produce assignment distributions over the same $K=64$ prototypes. Their weighted reconstructions form a structured representation for retrieval and image generation. Prototype contrast supplies the main cross-modal training signal, distribution alignment coordinates codebook use across modalities, an instance anchor preserves fine-grained discrimination, and diversity regularization prevents redundant prototypes. The shared assignment profile also makes the learned bottleneck directly testable through prototype retention, random selection, and identity shuffling. Across five seeds, ProtoEEG reaches 64.0\% Top-1 and 91.5\% Top-5 on THINGS-EEG2 200-way retrieval. After retraining on EEG-ImageNet, it obtains 44.8\% Top-1 and 73.8\% Top-5 in 80-way retrieval. ProtoEEG also ranks first on all seven reconstruction metrics. Removing prototype contrast lowers Top-1 by 56.9 percentage points, while shuffling prototype identities reduces it to 0.5\%. Prototype analyses further show distributed assignments and broad codebook use rather than one-prototype routing. These results support structured alignment targets as a practical alternative to increasing encoder or teacher capacity alone.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33065 [cs.CV] |
| (or arXiv:2610.33065v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33065 arXiv-issued DOI via DataCite |
Submission history
From: Kieren Yu [view email]
[v1]
Tue, 20 Oct 2026 07:56:51 UTC (4,496 KB)
来源：arXiv cs.LG + cs.RO + cs.HC · arxiv.org