Multi-task Alzheimer's Disease Classification Based on Adversarial Learning and Cross-attention

doi:10.11938/cjmr20253168

Abstract

Abstract:

Magnetic resonance imaging (MRI) and positron emission tomography (PET) are commonly used imaging techniques for the early diagnosis of Alzheimer's disease (AD). The combination of these two modalities enables a more comprehensive assessment of brain status by utilizing both anatomical and metabolic information. However, traditional multimodal fusion, which relies primarily on simple channel splicing, fails to fully exploit the complementary information across modalities and limits the model's effectiveness. To address this, this paper proposes a multi-task classification model for AD based on adversarial learning and cross-attention. The model reduces inter-modal feature discrepancies through adversarial learning, followed by feature fusion via cross-attention, and introduces a brain age prediction task as an auxiliary task to improve classification performance. Experimental results demonstrate that the proposed method achieves an accuracy of 91.10% and an F1 score of 91.01% in classifying AD, mild cognitive impairment (MCI), and normal controls (NC). This not only enhances the accuracy of early diagnosis but also strengthens the ability to monitor disease progression, thereby providing strong support for clinical interventions in AD.

Key words: adversarial learning, cross-attention mechanisms, multitasking, Alzheimer's disease, multimodality

CLC Number:

O482.53

GU Jiajia, WANG Yuanjun. Multi-task Alzheimer's Disease Classification Based on Adversarial Learning and Cross-attention[J]. Chinese Journal of Magnetic Resonance, 2026, 43(2): 186-199.

Figures/Tables 11

Fig. 1

Fig. 2

Fig. 3

Table 1

Table 2

Fig. 4

Fig. 5

Table 3

Fig. 6

Table 4

Table 5

References 27

[1]	ANDERSON N D. State of the science on mild cognitive impairment (MCI)[J]. CNS Spectrums, 2019, 24(1): 78-87. doi: 10.1017/S1092852918001347 pmid: 30651152
[2]	CUMMINGS J, ZHOU Y, LEE G, et al. Alzheimer's disease drug development pipeline: 2023[J]. Alzh Dement-TRCI, 2023, 9(2): e12385.
[3]	BEACH T G, MONSELL S E, Phillips L E, et al. Accuracy of the clinical diagnosis of Alzheimer disease at National Institute on Aging Alzheimer Disease Centers, 2005-2010[J]. J Neuropathol Exp, 2012, 71(4): 266-273.
[4]	LIU H, JIN F, Zeng H, et al. Image enhancement guided object detection in visually degraded scenes[J]. IEEE T Neur Net Lear, 2024, 35(10): 14164-14177.
[5]	KONG Z, ZHANG M, ZHU W, et al. Multi-modal data Alzheimer’s disease detection based on 3D convolution[J]. Biomed Signal Proces, 2022, 75: 103565. doi: 10.1016/j.bspc.2022.103565
[6]	ZHANG F, LI Z, ZHANGH B, et al. Multi-modal deep learning model for auxiliary diagnosis of Alzheimer’s disease[J]. Neurocomputing, 2019, 361: 185-195. doi: 10.1016/j.neucom.2019.04.093
[7]	MENG X, LIU J, FAN X, et al. Multi-modal neuroimaging neural network-based feature detection for diagnosis of Alzheimer’s disease[J]. Front Aging Neurosci, 2022, 14: 911220. doi: 10.3389/fnagi.2022.911220
[8]	KUN H A N, HAIWEI P A N, WEI Z, et al. Alzheimer's disease classification method based on multi-modal medical images[J]. J Tsinghua Univ (Sci Technol), 2020, 60(8): 664-671,682.
[9]	SONG J, ZHENG J, LI P, et al. An effective multimodal image fusion method using MRI and PET for Alzheimer's disease diagnosis[J]. Front Digit Health, 2021, 3: 637386. doi: 10.3389/fdgth.2021.637386
[10]	LOGAN R, WILLIAMS B G, FERREIRA DA SILVA M, et al. Deep convolutional neural networks with ensemble learning and generative adversarial networks for Alzheimer’s disease image data classification[J]. Front Aging Neurosci, 2021, 13: 720226. doi: 10.3389/fnagi.2021.720226
[11]	ZHANG Y X, WU X H, TANG L L, et al. Alzheimer's disease classification method based on multimodal data[J]. J Comput Appl, 2023, 43(S2): 298.
	张昀枭, 吴晓红, 唐荔莉, 等. 基于多模态数据的阿尔兹海默病分类方法[J]. 计算机应用, 2023, 43(S2): 298.
[12]	GU J J, WANG Y J. Hybrid attention and multiscale module for Alzheimer's disease classification[J]. Chinese J Magn Reson, 2025, 42(2): 103-116.
	顾佳佳, 王远军. 混合注意力和多尺度模块的阿尔茨海默病分类方法[J]. 波谱学杂志, 2025, 42(2): 103-116. doi: 10.11938/cjmr20243132
[13]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[14]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv (2021-06-03) [2025-05-30]. https://arxiv.org/abs/2010.11929.
[15]	ZHU J, TAN Y, LIN R, et al. Efficient self-attention mechanism and structural distilling model for Alzheimer’s disease diagnosis[J]. Comput Biol Med, 2022, 147: 105737. doi: 10.1016/j.compbiomed.2022.105737
[16]	KUSHOL R, MASOUMZADEH A, HUO D, et al. Addformer: Alzheimer’s disease detection from structural mri using fusion transformer[C]// 2022 IEEE 19th I Symp Biomed Imaging (ISBI). IEEE, 2022: 1-5.
[17]	KAUFMANN T, VAN DER MEER D, DOAN N T, et al. Common brain disorders are associated with heritable patterns of apparent aging of the brain[J]. Nat Neurosci, 2019, 22(10): 1617-1623. doi: 10.1038/s41593-019-0471-7 pmid: 31551603
[18]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// International Conference on Learning Representations (ICLR). 2015: 1-14.
[19]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems, 2014, 2: 2672-2680.
[20]	CHEN C F R, FAN Q, PANDA R. Crossvit: Cross-attention multi-scale vision transformer for image classification[C]// 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 2021: 347-356.
[21]	JACK JR C R, BERNSTEIN M A, FOX N C, et al. The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods[J]. J Magn Reson Imaging, 2008, 27(4): 685-691. doi: 10.1002/jmri.21049 pmid: 18302232
[22]	ESKILDSEN S F, COUPE P, GARCIA-LORENZO D, et al. Prediction of Alzheimer's disease in subjects with mild cognitive impairment from the ADNI cohort using patterns of cortical thinning[J]. NeuroImage, 2013, 65: 511-521. doi: 10.1016/j.neuroimage.2012.09.058 pmid: 23036450
[23]	JENKINSON M, BECKMANe C F, BEHRENS T E J, et al. FSL[J]. NeuroImage, 2012, 62(2): 782-790. doi: 10.1016/j.neuroimage.2011.09.015 pmid: 21979382
[24]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization[C]// 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 618-626.
[25]	HUANG Y, XU J, ZHOU Y, et al. Diagnosis of Alzheimer’s disease via multi-modality 3D convolutional neural network[J]. Front Neurosci, 2019, 13: 509. doi: 10.3389/fnins.2019.00509
[26]	ZHANG M, SUN L, KONG Z, et al. Pyramid-attentive GAN for multimodal brain image complementation in Alzheimer’s disease classification[J]. Biomed Signal Process Control, 2024, 89: 105652. doi: 10.1016/j.bspc.2023.105652
[27]	ABUHMED T, El-SAPPAGH S, AlONSON J M. Robust hybrid deep learning models for Alzheimer’s progression detection[J]. Knowl-Based Syst, 2021, 213: 106688. doi: 10.1016/j.knosys.2020.106688

分组	样本数量	MAE
AD	210	6.34
MCI	257	5.71
NC	267	3.52

模态	组合			ACC/(%)	F1/(%)	SEN/(%)	SPE/(%)	PRE/(%)
模态	AL	MCFM	MT	ACC/(%)	F1/(%)	SEN/(%)	SPE/(%)	PRE/(%)
MRI				74.79(±1.46)	72.38	73.39	87.38	75.31
PET				80.27(±2.65)	78.27	78.77	88.62	82.41
MRI+PET				85.18(±2.59)	83.15	83.60	91.61	83.16
MRI+PET+age	√		√	87.66(±2.48)	87.66	88.09	93.14	87.61
MRI+PET+age	√	√		90.00(±2.23)	89.50	89.26	94.95	90.28
MRI+PET+age		√	√	88.51(±2.45)	87.04	87.80	93.44	89.99
MRI+PET+age	√	√	√	91.10(±1.79)	91.01	90.53	95.53	90.90

方法	模态		ACC/(%)	F1/(%)	SEN/(%)	SPE/(%)	PRE/(%)
方法	MRI	PET	ACC/(%)	F1/(%)	SEN/(%)	SPE/(%)	PRE/(%)
SACNet	√		70.30(±3.66)	69.91	64.49	77.99	79.67
SACNet		√	73.33(±3.65)	73.04	76.11	70.00	75.49
MACNet	√	√	76.53(±2.85)	76.36	85.00	67.33	75.82

模型	ACC/(%)	F1/(%)	SEN/(%)	SPE/(%)	PRE/(%)
3D VGG16	85.18(±2.59)	83.15	83.60	91.61	83.16
3D ResNet34	86.30(±2.41)	86.41	86.78	93.14	86.24
3D ViT	76.71(±2.63)	76.28	76.62	88.61	76.64
MACNet (本文)	91.10(±1.79)	91.01	90.53	95.53	90.90

文献	图像类型	分类模型	类别	准确率/（%）
Huang等^[25]	MRI+PET	VGG	1211例	90.10
Song等^[9]	MRI+PET	3D Simple CNN	95 AD/160 MCI/126 NC	74.54
Zhang等^[26]	MRI+PET	PA-Net	370例	89.90
Abuhmed等^[27]	MRI+PET+神经心理学+神经病理学+认知评分	BiLSTM+随机森林（多任务）	1371例	84.95
本文	MRI+PET+年龄	MACNet	210 AD/257 MCI/267 NC	91.10