用于超快时空编码MRI的Transformer超分辨率重建算法研究
收稿日期: 2024-04-16
网络出版日期: 2024-05-16
基金资助
国家重点研发计划(2023YFE0113300);国家重点研发计划(2022YFF0707000);国家自然科学基金项目(22327901);中国科学院B类战略性先导科技专项(XDB0540300);湖北省科技创新人才及服务专项(2023EHA003);中国科学院磁共振技术联盟科研仪器设备研制项目(2021GZL001);中国科学院精密测量科学与技术创新研究院交叉培养项目(S21S4101);中国科学院科研仪器开发项目(YJKYYQ 20190032)
Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging
Received date: 2024-04-16
Online published: 2024-05-16
时空编码（SPEN）磁共振成像（MRI）是一种超快MRI技术，通过该技术采集获得的原始图像空间分辨率较低，需要基于序列物理原理进行超分辨率重建以提高其原始图像的分辨率，而现有的基于深度学习SPEN超分辨率重建算法在提取图像像素长距离依赖关系上的能力有限.为了解决此问题，本文提出了一种基于Transformer的SPEN MRI超分辨率重建算法.该算法采用编码器-解码器结构，并引入Transformer模块以提取特征图的局部上下文信息和长距离依赖关系.实验结果表明，本文所提的重建算法可以在不增加额外采样点的情况下从SPEN低分辨率图像中重建出高空间分辨率、无混叠伪影的超分辨率图像.与现有的超分辨率算法相比，本文提出的算法在临床前以及临床数据集上都取得了更好的重建效果.
宁欣宙 , 黄臻 , 陈西曲 , 刘鑫杰 , 陈罡 , 张志 , 鲍庆嘉 , 刘朝阳 . 用于超快时空编码MRI的Transformer超分辨率重建算法研究[J]. 波谱学杂志, 2024 , 41(4) : 454 -468 . DOI: 10.11938/cjmr20243110
Spatio-temporal encoding (SPEN) magnetic resonance imaging (MRI) is an ultrafast MRI technique. However, resolution of the original image acquired with SPEN is relatively low, requiring super-resolution reconstruction based on sequence physics principles to improve spatial resolution. As the existing SPEN super-resolution reconstruction algorithms based on deep learning have confined abilities to capture long-range dependencies, this paper proposes a transformer-based SPEN MRI super-resolution reconstruction algorithm. An encoder-decoder structure is adopted, and a transformer module is introduced to extract local context information and long-range dependencies of feature maps. Experimental results show that the proposed reconstruction method can reconstruct a super-resolution image with high spatial resolution and no aliasing artifacts from the low-resolution SPEN image without adding additional sampling points. Compared to the existing super-resolution methods, the proposed method achieves better results on both clinical and preclinical datasets.
| [1] | EMAM M M, HOUSSEIN E H, GHONIEM R M. A modified reptile search algorithm for global optimization and image segmentation: Case study brain MRI images[J]. Comput Biol Med, 2023, 152: 106404. |
| [2] | WALTER S S, FRITZ B, KIJOWSKI R, et al. 2D versus 3D MRI of osteoarthritis in clinical practice and research[J]. Skeletal Radiol, 2023, 52(11): 2211-2224. |
| [3] | TSAO J. Ultrafast imaging: principles, pitfalls, solutions, and applications[J]. J Magn Reson Imaging, 2010, 32(2): 252-266. |
| [4] | GIL R, FERNANDES F F, SHEMESH N. Neuroplasticity-driven timing modulations revealed by ultrafast functional magnetic resonance imaging[J]. NeuroImage, 2021, 225: 117446. |
| [5] | HIRSCH F W, FRAHM J, SORGE I, et al. Real-time magnetic resonance imaging in pediatric radiology—new approach to movement and moving children[J]. Pediatr Radiol, 2021, 51: 840-846. |
| [6] | MILON A, FLAMENT V, GUENICHE Y, et al. How to optimize breast MRI protocol? The value of combined analysis of ultrafast and diffusion-weighted MRI sequences[J]. Diagn Interv Imaging, 2023, 104(6): 284-291. |
| [7] | LIAO C, CAO X, CHO J, et al. Highly efficient MRI through multi-shot echo planar imaging[C]// Wavelets and Sparsity XVIII, 2019, 11138: 353-365. |
| [8] | DUONG S T, PHUNG S L, BOUZERDOUM A, et al. An unsupervised deep learning technique for susceptibility artifact correction in reversed phase-encoding EPI images[J]. Magn Reson Imaging, 2020, 71: 1-10. |
| [9] | SHROT Y, FRYDMAN L. Spatially encoded NMR and the acquisition of 2D magnetic resonance images within a single scan[J]. J Magn Reson, 2005, 172(2): 179-190. |
| [10] | CAI S H, CAI C B, CHEN Z. Spatiotemporally encoded nuclear magnetic resonance and its applications[J]. J Xiamen University(Natural Science), 2021, 60(3): 542-554. |
| 蔡淑惠, 蔡聪波, 陈忠. 时空编码核磁共振方法及其应用[J]. 厦门大学学报(自然科学版), 2021, 60(3): 542-554. | |
| [11] | TAL A, FRYDMAN L. Spatial encoding and the single-scan acquisition of high definition MR images in inhomogeneous fields[J]. J Magn Reson, 2006, 182(2): 179-194. |
| [12] | TAL A, FRYDMAN L. Single-scan multidimensional magnetic resonance[J]. Prog Nucl Magn Reson Spectrosc, 2010, 57(3): 241-292. |
| [13] | BEN-ELIEZER N, IRANI M, FRYDMAN L. Super-resolved spatially encoded single-scan 2D MRI[J]. Magn Reson Med, 2010, 63(6): 1594-1600. |
| [14] | CHEN Y, LI J, QU X, et al. Partial Fourier transform reconstruction for single-shot MRI with linear frequency-swept excitation[J]. Magn Reson Med, 2013, 69(5): 1326-1336. |
| [15] | CAI C, DONG J, CAI S, et al. An efficient de-convolution reconstruction method for spatiotemporal-encoding single-scan 2D MRI[J]. J Magn Reson, 2013, 228: 136-147. |
| [16] | CHEN L, LI J, ZHANG M, et al. Super-resolved enhancing and edge deghosting (SEED) for spatiotemporally encoded single-shot MRI[J]. Med Image Anal, 2015, 23(1): 1-14. |
| [17] | ZHONG S, CHEN M, WEI X, et al. Understanding aliasing effects and their removal in SPEN MRI: A k-space perspective[J]. Magn Reson Med, 2023, 90(1): 166-176. |
| [18] | LI Y J, YANG X Y, YANG X M. Magnetic resonance image reconstruction of multi-scale residual unet fused with attention mechanism[J]. Chinese J Magn Reson, 2023, 40(3):307-319. |
| 李奕洁, 杨馨雨, 杨晓梅. 融合注意力机制的多尺度残差Unet的磁共振图像重建[J]. 波谱学杂志, 2023, 40(3): 307-319. | |
| [19] | LU Q Q, LIAN Z F, LI J L, et al. Magnetic resonance R2* parameter mapping of liver based on self-supervised deep neural network[J]. Chinese J Magn Reson, 2023, 40(3):258-269. |
| 陆琪琪, 连梓锋, 李嘉龙. 等. 基于自监督网络的肝脏磁共振R2*参数图像重建[J]. 波谱学杂志, 2023, 40(3): 258-269. | |
| [20] | DE LEEUW DEN BOUTER M L, IPPOLITO G, O’REILLY T P, et al. Deep learning-based single image super-resolution for low-field MR brain images[J]. Sci Rep, 2022, 12(1): 6362. |
| [21] | SONG L, WANG Q, LIU T, et al. Deep robust residual network for super-resolution of 2D fetal brain MRI[J]. Sci Rep, 2022, 12(1): 406. |
| [22] | XIANG P C, CAI C B, WANG J C, et al. Super-resolved reconstruction method for spatiotemporally encoded magnetic resonance imaging based on deep neural network>[J]. Acta Phys Sin, 2022, 71(5):363-371. |
| 向鹏程, 蔡聪波, 王杰超. 等. 基于深度神经网络的时空编码磁共振成像超分辨率重建方法[J]. 物理学报, 2022, 71(5): 363-371. | |
| [23] | CHEN X, WANG W, HUANG J, et al. Ultrafast water-fat separation using deep learning-based single-shot MRI[J]. Magn Reson Med, 2022, 87(6): 2811-2825. |
| [24] | CHOI S R, LEE M. Transformer architecture and attention mechanisms in genome data analysis: a comprehensive review[J]. Biology, 2023, 12(7): 1033. |
| [25] | RAHALI A, AKHLOUFI M A. End-to-end transformer-based models in textual-based NLP[J]. AI, 2023, 4(1): 54-110. |
| [26] | YIN Y, TANG Z, WENG H. Application of visual transformer in renal image analysis[J]. BioMed Eng OnLine, 2024, 23(1): 27. |
| [27] | ZU B, CAO T, LI Y, et al. SwinT-SRNet: Swin transformer with image super-resolution reconstruction network for pollen images classification[J]. Eng Appl Artif Intell, 2024, 133: 108041. |
| [28] | DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]// International Conference on Learning Representations, 2021. |
| [29] | LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// IEEE/CVF International Conference on Computer Vision, 2021: 9992-10002. |
| [30] | CHEN H, WANG Y, GUO T, et al. Pre-trained image processing transformer[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12294-12305. |
| [31] | LIANG J, CAO J, SUN G, et al. SwinIR: Image restoration using swin transformer[C]// IEEE/CVF International Conference on Computer Vision Workshops, 2021: 833-1844. |
| [32] | ZAMIR S W, ARORA A, KHAN S, et al. Restormer: Efficient transformer for high-resolution image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5718-5729. |
| [33] | WANG Z, CUN X, BAO J, et al. Uformer: A general u-shaped transformer for image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 17662-17672. |
| [34] | SCHMIDT R, SEGINER A, FRYDMAN L. Interleaved multishot imaging by spatiotemporal encoding: A fast, self-referenced method for high-definition diffusion and functional MRI[J]. Magn Reson Med, 2016, 75(5): 1935-1948. |
| [35] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Neural Information Processing Systems, 2017: 6000-6010. |
| [36] | WU H, XIAO B, CODELLA N, et al. Cvt: Introducing convolutions to vision transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 22-31. |
| [37] | YUAN K, GUO S, LIU Z, et al. Incorporating convolution designs into visual transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 559-568. |
| [38] | VAN ESSEN D C, SMITH S M, BARCH D M, et al. The WU-Minn human connectome project: an overview[J]. NeuroImage, 2013, 80: 62-79. |
/
| 〈 |
|
〉 |