HeteroMetaNet: A multimodal meta-learning framework for ischemic stroke prediction from heterogeneous clinical and radiological data

Abstract

Accurate and timely stroke detection remains a major clinical challenge due to the heterogeneous and multimodal nature of patient data. Most existing deep learning approaches rely solely on either imaging or structured clinical records, which limits their robustness and generalizability in real-world settings. This study introduces two multimodal architectures, HeteroFuseNet and HeteroMetaNet, designed to integrate non-contrast CT brain images with structured electronic health record (EHR) variables within a unified framework, referred to as HeteroStroke. HeteroFuseNet employs a PCA-enhanced mid-level fusion strategy to learn joint representations of deep visual features extracted using ResNet-50, EfficientNet-B0, and Swin Transformer, combined with structured clinical embeddings. In contrast, HeteroMetaNet adopts a lightweight meta-learning approach that adaptively integrates modality-specific predictions from these vision models alongside clinical features to achieve robust, context-aware stroke classification. Experimental results demonstrate that both proposed models outperform image-only baselines, particularly in clinically heterogeneous cases, highlighting the effectiveness of multimodal fusion in enhancing diagnostic reliability. This work underscores the potential of cross-modal learning to support precision medicine in stroke care. Future work will extend the framework toward differentiating ischemic and hemorrhagic stroke subtypes for improved clinical decision support.