Chinese Journal of Magnetic Resonance >
Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging
Received date: 2024-04-16
Online published: 2024-05-16
Spatio-temporal encoding (SPEN) magnetic resonance imaging (MRI) is an ultrafast MRI technique. However, resolution of the original image acquired with SPEN is relatively low, requiring super-resolution reconstruction based on sequence physics principles to improve spatial resolution. As the existing SPEN super-resolution reconstruction algorithms based on deep learning have confined abilities to capture long-range dependencies, this paper proposes a transformer-based SPEN MRI super-resolution reconstruction algorithm. An encoder-decoder structure is adopted, and a transformer module is introduced to extract local context information and long-range dependencies of feature maps. Experimental results show that the proposed reconstruction method can reconstruct a super-resolution image with high spatial resolution and no aliasing artifacts from the low-resolution SPEN image without adding additional sampling points. Compared to the existing super-resolution methods, the proposed method achieves better results on both clinical and preclinical datasets.
NING Xinzhou , HUANG Zhen , CHEN Xiqu , LIU Xinjie , CHEN Gang , ZHANG Zhi , BAO Qingjia , LIU Chaoyang . Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging[J]. Chinese Journal of Magnetic Resonance, 2024 , 41(4) : 454 -468 . DOI: 10.11938/cjmr20243110
| [1] | EMAM M M, HOUSSEIN E H, GHONIEM R M. A modified reptile search algorithm for global optimization and image segmentation: Case study brain MRI images[J]. Comput Biol Med, 2023, 152: 106404. |
| [2] | WALTER S S, FRITZ B, KIJOWSKI R, et al. 2D versus 3D MRI of osteoarthritis in clinical practice and research[J]. Skeletal Radiol, 2023, 52(11): 2211-2224. |
| [3] | TSAO J. Ultrafast imaging: principles, pitfalls, solutions, and applications[J]. J Magn Reson Imaging, 2010, 32(2): 252-266. |
| [4] | GIL R, FERNANDES F F, SHEMESH N. Neuroplasticity-driven timing modulations revealed by ultrafast functional magnetic resonance imaging[J]. NeuroImage, 2021, 225: 117446. |
| [5] | HIRSCH F W, FRAHM J, SORGE I, et al. Real-time magnetic resonance imaging in pediatric radiology—new approach to movement and moving children[J]. Pediatr Radiol, 2021, 51: 840-846. |
| [6] | MILON A, FLAMENT V, GUENICHE Y, et al. How to optimize breast MRI protocol? The value of combined analysis of ultrafast and diffusion-weighted MRI sequences[J]. Diagn Interv Imaging, 2023, 104(6): 284-291. |
| [7] | LIAO C, CAO X, CHO J, et al. Highly efficient MRI through multi-shot echo planar imaging[C]// Wavelets and Sparsity XVIII, 2019, 11138: 353-365. |
| [8] | DUONG S T, PHUNG S L, BOUZERDOUM A, et al. An unsupervised deep learning technique for susceptibility artifact correction in reversed phase-encoding EPI images[J]. Magn Reson Imaging, 2020, 71: 1-10. |
| [9] | SHROT Y, FRYDMAN L. Spatially encoded NMR and the acquisition of 2D magnetic resonance images within a single scan[J]. J Magn Reson, 2005, 172(2): 179-190. |
| [10] | CAI S H, CAI C B, CHEN Z. Spatiotemporally encoded nuclear magnetic resonance and its applications[J]. J Xiamen University(Natural Science), 2021, 60(3): 542-554. |
| 蔡淑惠, 蔡聪波, 陈忠. 时空编码核磁共振方法及其应用[J]. 厦门大学学报(自然科学版), 2021, 60(3): 542-554. | |
| [11] | TAL A, FRYDMAN L. Spatial encoding and the single-scan acquisition of high definition MR images in inhomogeneous fields[J]. J Magn Reson, 2006, 182(2): 179-194. |
| [12] | TAL A, FRYDMAN L. Single-scan multidimensional magnetic resonance[J]. Prog Nucl Magn Reson Spectrosc, 2010, 57(3): 241-292. |
| [13] | BEN-ELIEZER N, IRANI M, FRYDMAN L. Super-resolved spatially encoded single-scan 2D MRI[J]. Magn Reson Med, 2010, 63(6): 1594-1600. |
| [14] | CHEN Y, LI J, QU X, et al. Partial Fourier transform reconstruction for single-shot MRI with linear frequency-swept excitation[J]. Magn Reson Med, 2013, 69(5): 1326-1336. |
| [15] | CAI C, DONG J, CAI S, et al. An efficient de-convolution reconstruction method for spatiotemporal-encoding single-scan 2D MRI[J]. J Magn Reson, 2013, 228: 136-147. |
| [16] | CHEN L, LI J, ZHANG M, et al. Super-resolved enhancing and edge deghosting (SEED) for spatiotemporally encoded single-shot MRI[J]. Med Image Anal, 2015, 23(1): 1-14. |
| [17] | ZHONG S, CHEN M, WEI X, et al. Understanding aliasing effects and their removal in SPEN MRI: A k-space perspective[J]. Magn Reson Med, 2023, 90(1): 166-176. |
| [18] | LI Y J, YANG X Y, YANG X M. Magnetic resonance image reconstruction of multi-scale residual unet fused with attention mechanism[J]. Chinese J Magn Reson, 2023, 40(3):307-319. |
| 李奕洁, 杨馨雨, 杨晓梅. 融合注意力机制的多尺度残差Unet的磁共振图像重建[J]. 波谱学杂志, 2023, 40(3): 307-319. | |
| [19] | LU Q Q, LIAN Z F, LI J L, et al. Magnetic resonance R2* parameter mapping of liver based on self-supervised deep neural network[J]. Chinese J Magn Reson, 2023, 40(3):258-269. |
| 陆琪琪, 连梓锋, 李嘉龙. 等. 基于自监督网络的肝脏磁共振R2*参数图像重建[J]. 波谱学杂志, 2023, 40(3): 258-269. | |
| [20] | DE LEEUW DEN BOUTER M L, IPPOLITO G, O’REILLY T P, et al. Deep learning-based single image super-resolution for low-field MR brain images[J]. Sci Rep, 2022, 12(1): 6362. |
| [21] | SONG L, WANG Q, LIU T, et al. Deep robust residual network for super-resolution of 2D fetal brain MRI[J]. Sci Rep, 2022, 12(1): 406. |
| [22] | XIANG P C, CAI C B, WANG J C, et al. Super-resolved reconstruction method for spatiotemporally encoded magnetic resonance imaging based on deep neural network>[J]. Acta Phys Sin, 2022, 71(5):363-371. |
| 向鹏程, 蔡聪波, 王杰超. 等. 基于深度神经网络的时空编码磁共振成像超分辨率重建方法[J]. 物理学报, 2022, 71(5): 363-371. | |
| [23] | CHEN X, WANG W, HUANG J, et al. Ultrafast water-fat separation using deep learning-based single-shot MRI[J]. Magn Reson Med, 2022, 87(6): 2811-2825. |
| [24] | CHOI S R, LEE M. Transformer architecture and attention mechanisms in genome data analysis: a comprehensive review[J]. Biology, 2023, 12(7): 1033. |
| [25] | RAHALI A, AKHLOUFI M A. End-to-end transformer-based models in textual-based NLP[J]. AI, 2023, 4(1): 54-110. |
| [26] | YIN Y, TANG Z, WENG H. Application of visual transformer in renal image analysis[J]. BioMed Eng OnLine, 2024, 23(1): 27. |
| [27] | ZU B, CAO T, LI Y, et al. SwinT-SRNet: Swin transformer with image super-resolution reconstruction network for pollen images classification[J]. Eng Appl Artif Intell, 2024, 133: 108041. |
| [28] | DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]// International Conference on Learning Representations, 2021. |
| [29] | LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// IEEE/CVF International Conference on Computer Vision, 2021: 9992-10002. |
| [30] | CHEN H, WANG Y, GUO T, et al. Pre-trained image processing transformer[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12294-12305. |
| [31] | LIANG J, CAO J, SUN G, et al. SwinIR: Image restoration using swin transformer[C]// IEEE/CVF International Conference on Computer Vision Workshops, 2021: 833-1844. |
| [32] | ZAMIR S W, ARORA A, KHAN S, et al. Restormer: Efficient transformer for high-resolution image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5718-5729. |
| [33] | WANG Z, CUN X, BAO J, et al. Uformer: A general u-shaped transformer for image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 17662-17672. |
| [34] | SCHMIDT R, SEGINER A, FRYDMAN L. Interleaved multishot imaging by spatiotemporal encoding: A fast, self-referenced method for high-definition diffusion and functional MRI[J]. Magn Reson Med, 2016, 75(5): 1935-1948. |
| [35] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Neural Information Processing Systems, 2017: 6000-6010. |
| [36] | WU H, XIAO B, CODELLA N, et al. Cvt: Introducing convolutions to vision transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 22-31. |
| [37] | YUAN K, GUO S, LIU Z, et al. Incorporating convolution designs into visual transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 559-568. |
| [38] | VAN ESSEN D C, SMITH S M, BARCH D M, et al. The WU-Minn human connectome project: an overview[J]. NeuroImage, 2013, 80: 62-79. |
/
| 〈 |
|
〉 |