Magnetic resonance imaging (MRI) and positron emission tomography (PET) are commonly used imaging techniques for the early diagnosis of Alzheimer's disease (AD). The combination of these two modalities enables a more comprehensive assessment of brain status by utilizing both anatomical and metabolic information. However, traditional multimodal fusion, which relies primarily on simple channel splicing, fails to fully exploit the complementary information across modalities and limits the model's effectiveness. To address this, this paper proposes a multi-task classification model for AD based on adversarial learning and cross-attention. The model reduces inter-modal feature discrepancies through adversarial learning, followed by feature fusion via cross-attention, and introduces a brain age prediction task as an auxiliary task to improve classification performance. Experimental results demonstrate that the proposed method achieves an accuracy of 91.10% and an F1 score of 91.01% in classifying AD, mild cognitive impairment (MCI), and normal controls (NC). This not only enhances the accuracy of early diagnosis but also strengthens the ability to monitor disease progression, thereby providing strong support for clinical interventions in AD.