Low-field nuclear magnetic resonance (LF-NMR) relaxation time spectrum analysis has been widely applied in petroleum engineering and geological exploration. However, conventional NMR relaxation time spectrum inversion algorithms require time-domain signals with high signal-to-noise ratios (SNR), which are often difficult to obtain in practical scenarios such as on-site core analysis. To achieve accurate and stable inversion of low-SNR NMR signals, this paper proposes a deep learning-based inversion network named UDMCA (U-Net Denoising and Multi-scale Cross Attention Inversion Network), which incorporates a multi-scale cross-attention mechanism to extract and fuse relaxation features at multiple scales according to the decay characteristics of NMR signals, thereby improving inversion accuracy for low-SNR multi-component signals. In addition, a U-Net denoising module is introduced to further enhance noise robustness. Experimental results show that the proposed method achieves accurate and stable inversion for low-SNR signals and demonstrates good applicability in core NMR signal inversion, providing an effective solution for low-SNR LF-NMR data processing and inversion.