arXiv cs.LG + cs.RO + cs.HC· Hanning Guo, Hanwen Bi, Farah Abdellatif, Andrei Galbenus, Jon. N. Shah, Abigail Morrison, J\"urgen Dammers·· 2 天前精选AI 评分62
Brain-OF：联合预训练 fMRI、EEG 和 MEG 的全功能脑基础模型
Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG
AI 导读
研究团队提出 Brain-OF，一个在 fMRI、EEG 和 MEG 上联合预训练的全功能脑基础模型，可在统一框架内处理单模态和多模态输入。
推荐理由
该预印本把 fMRI、EEG 和 MEG 放进同一个基础模型，并公开代码与权重，读者可据此判断多模态神经影像预训练的整合路径。
正文 · 原文
Abstract:Brain foundation models have achieved remarkable advances across a wide range of neuroscience tasks. However, most existing models are limited to a single functional modality, restricting their ability to exploit complementary spatiotemporal dynamics and the collective data scale across different neuroimaging techniques. This limitation largely arises from severe semantic heterogeneity and resolution discrepancies among modalities. To address these challenges, we propose Brain-OF, an omnifunctional brain foundation model jointly pretrained on fMRI, EEG and MEG, capable of handling both unimodal and multimodal inputs within a unified framework. To reconcile heterogeneous spatiotemporal resolutions, we introduce the Any-Resolution Neural Signal Sampler, which projects diverse brain signals into a shared semantic space. To further manage semantic shifts, the Brain-OF backbone integrates DINT attention with a Sparse Mixture of Experts, where shared experts capture modality-invariant representations and routed experts specialize in modality-specific semantics. Furthermore, to explicitly internalize the characteristics of neural activity through self-supervised learning, we propose Masked Temporal-Frequency Modeling, a dual-domain pretraining objective that jointly reconstructs brain signals in both the time and frequency domains. Brain-OF is pretrained on a large-scale corpus comprising around 40 datasets and demonstrates superior performance across diverse downstream tasks, highlighting the benefits of joint multimodal integration and dual-domain pretraining.
| Comments: | Accepted at NeurIPS 2026. Code and pretrained model weights are available on our project page: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Signal Processing (eess.SP); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2602.23410 [cs.LG] |
| (or arXiv:2602.23410v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.23410 arXiv-issued DOI via DataCite |
Submission history
From: Hanning Guo [view email]
[v1]
Thu, 26 Feb 2026 15:47:13 UTC (5,046 KB)
[v2]
Mon, 2 Mar 2026 10:08:49 UTC (5,046 KB)
[v3]
Fri, 15 May 2026 11:39:26 UTC (5,114 KB)
[v4]
Sun, 18 Oct 2026 14:53:20 UTC (4,484 KB)
来源：arXiv cs.LG + cs.RO + cs.HC · arxiv.org