Chinese Journal of Magnetic Resonance >

2022 , Vol. 39 >Issue 3: 303 - 315

DOI: https://doi.org/10.11938/cjmr20222988

Articles

Automatic Segmentation of Knee Joint Synovial Magnetic Resonance Images Based on 3D VNetTrans

  • Ying-shan WANG ,
  • Ao-qi DENG ,
  • Jin-ling MAO ,
  • Zhong-qi ZHU ,
  • Jie SHI ,
  • Guang YANG ,
  • Wei-wei MA ,
  • Qing LU ,
  • Hong-zhi WANG
Expand
  • 1. Shanghai Key Laboratory of Magnetic Resonance, School of Physics and Electronic Science, East China Normal University, Shanghai 200062, China
    2. Shanghai Guanghua Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai 200052; China
    3. College of Acupuncture and Massage, Shanghai University of Chinese Medicine, Shanghai 200032, China
    4. Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai 200127, China

Received date: 2022-03-23

  Online published: 2022-05-11

Fold

Abstract

Knee joint is commonly hurt by rheumatoid arthritis (RA). Accurate segmentation of synovium is essential for the diagnosis and treatment of RA. This paper proposes an algorithm based on improved VNet for automatically segmenting knee joint synovial magnetic resonance images. Firstly, the knee joint magnetic resonance images of 39 patients with synovitis were preprocessed. VNetTrans was constructed by embedding Transformer at the bottom of VNet. The MemSwish activation function was used for training. The average Dice score of the final model is 0.758 5 and the HD is 24.6 mm. Compared with VNet, the proposed model increased Dice score by 0.083 6 and decreased HD by 10 mm. Experimental results demonstrated that the proposed algorithm achieved satisfying 3D segmentation of the synovial hyperplasia area in the knee magnetic resonance images. It can be utilized to facilitate the diagnosis and monitoring of RA.

Key words： magnetic resonance image; medical image segmentation; deep learning; synovitis

Cite this article

Ying-shan WANG , Ao-qi DENG , Jin-ling MAO , Zhong-qi ZHU , Jie SHI , Guang YANG , Wei-wei MA , Qing LU , Hong-zhi WANG . Automatic Segmentation of Knee Joint Synovial Magnetic Resonance Images Based on 3D VNetTrans[J]. Chinese Journal of Magnetic Resonance, 2022 , 39(3) : 303 -315 . DOI: 10.11938/cjmr20222988

References

1 SMOLEN J S, ALETAHA D, BARTON A, et alRheumatoid arthritis[J]. Nat Rev Dis Primers, 2018,4 (1): 18001.
2 JIN S Y, LI M T, FANG Y F, et alChinese registry of rheumatoid arthritis (CREDIT): II. prevalence and risk factors of major comorbidities in Chinese patients with rheumatoid arthritis[J]. Arthritis ResTher, 2017,19 (1): 251.
3 中华医学会风湿病学分会2018中国类风湿关节炎诊疗指南[J]. 中华内科杂志, 2018,57 (4): 242- 251.
3 ASSOCIATION C R2018 Chinese guideline for the diagnosis and treatment of rheumatoid arthritis[J]. Chin J Intern Med, 2018,57 (4): 242- 251.
4 SUGIMOTO H, TAKEDA A, KANO SAssessment of disease activity in rheumatoid arthritis using magnetic resonance imaging: quantification of pannus volume in the hands[J]. Bri J Rheumatol, 1998,37 (8): 854- 861.
5 OSTERGAARD MDifferent approaches to synovial membrane volume determination by magnetic resonance imaging: manual versus automated segmentation[J]. Rheumatology, 1997,36 (11): 1166- 1177.
6 SAKASHITA T, KAMISHIMA T, KOBAYASHI Y, et alAccurate quantitative assessment of synovitis in rheumatoid arthritis using pixel-by-pixel, time-intensity curve shape analysis[J]. Br J Radiol, 2016,89 (1061): 20151000.
7 FOTINOS-HOYER A K, GUERMAZI A, JARA H, et alAssessment of synovitis in the osteoarthritic knee: comparison between manual segmentation, semiautomated segmentation, and semiquantitative assessment using contrast-enhanced fat-suppressed T1-weighted MRI[J]. Magn Reson Med, 2010,64 (2): 604- 609.
8 PERRY T A, GAIT A, O’NEILL T W, et alMeasurement of synovial tissue volume in knee osteoarthritis using a semiautomated MRI-based quantitative approach[J]. Magn Reson Med, 2019,81 (5): 3056- 3064.
9 WANG A, FRANKE A, WESARG S. Semi-automatic segmentation of JIA-induced inflammation in MRI images of ankle joints[C]// Medical Imaging 2019: Image Processing, SPIE, 2019, 10949: 875-881.
10 ANDERSEN J K H, PEDERSEN J S, LAURSEN M S, et alNeural networks for automatic scoring of arthritis disease activity on ultrasound images[J]. RMD open, 2019,5 (1): e000891.
11 CHRISTENSEN A B H, JUST S A, ANDERSEN J K H, et alApplying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients[J]. Ann Rheum Dise, 2020,79 (9): 1189- 1193.
12 IQBAL I, SHAHZAD G, RAFIQ N, et alDeep learning-based automated detection of human knee joint's synovial fluid from magnetic resonance images with transfer learning[J]. IET Image Processing, 2020,14 (10): 1990- 1998.
13 WONG L M, SHI L, XIAO F, et alFully automated segmentation of wrist bones on T2-weighted fat-suppressed MR images in early rheumatoid arthritis[J]. Quant Imag Med Surg, 2019,9 (4): 579.
14 魏小娜, 邢嘉祺, 王振宇, 等基于改进U-Net的关节滑膜磁共振图像的分割[J]. 计算机应用, 2020,40 (11): 3340- 3345.
14 WEI X N, XIN J Q, WANG Z Y, et alMagnetic resonance image segmentation of articular synovium based on improved U-Net[J]. Journal of Computer Applications, 2020,40 (11): 3340- 3345.
15 王振宇, 王颖珊, 毛瑾玲, 等基于Dense-UNet++的关节滑膜磁共振图像分割[J]. 波谱学杂志, 2022,39 (2): 208- 219.
15 WANG Z Y, WANG Y S, MAO J L, et alMagnetic resonance images segmentation of synovium based on Dense-UNet++[J]. Chinese J Magn Reson, 2022,39 (2): 208- 219.
16 WANG W X, CHEN C, DING M, et al. TransBTS: Multimodal brain tumor segmentation using transformer[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2021: 109-119.
17 LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2015: 3431-3440.
18 RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]// International Conference on Medical image computing and computer-assisted intervention, Cham: Springer, 2015: 234-241.
19 ?I?EK ?, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]// International conference on medical image computing and computer-assisted intervention. Springer, 2016: 424-432.
20 MILLETARI F, NAVAB N, AHMADI S-A. V-net: Fully convolutional neural networks for volumetric medical image segmentation[C]// 2016 fourth international conference on 3D vision (3DV), IEEE, 2016: 565-571.
21 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the Advances in Neural Information Processing Systems, 2017: 6000-6010.
22 DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[OL]. arXiv preprint arXiv: 2010.11929, 2020.
23 NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]// Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel: 2010: 807-814.
24 RAMACHANDRAN P, ZOPH B, LE Q V. Searching for activation functions[J]. arXiv preprint arXiv: 1710.05941, 2017.
25 TAN M, LE Q. Efficientnet: Rethinking model scaling for convolutional neural networks[C]// International conference on machine learning, PMLR, 2019: 6105-6114.
26 YUAN L, CHEN Y, WANG T, et al. Tokens-to-token vit: Training vision transformers from scratch on imagenet[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 558-567.
27 YUSHKEVICH P A, GAO Y, GERIG G. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images[C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2016: 3342-3345.
28 HATAMIZADEH A, TANG Y, NATH V, et al. Unetr: Transformers for 3d medical image segmentation[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022: 574-584.
Options
Outlines

/

〈 〉

Copyright © 2017 Editorial Board of Chinese Journal of Magnetic Resonance
Powered by Beijing Magtech Co. Ltd