Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Aug;39(8):2664-2675.
doi: 10.1109/TMI.2020.2995108.

Relational Modeling for Robust and Efficient Pulmonary Lobe Segmentation in CT Scans

Weiyi Xie, Colin Jacobs, Jean-Paul Charbonnier, Bram van Ginneken

Relational Modeling for Robust and Efficient Pulmonary Lobe Segmentation in CT Scans

Weiyi Xie et al. IEEE Trans Med Imaging. 2020 Aug.

Abstract

Pulmonary lobe segmentation in computed tomography scans is essential for regional assessment of pulmonary diseases. Recent works based on convolution neural networks have achieved good performance for this task. However, they are still limited in capturing structured relationships due to the nature of convolution. The shape of the pulmonary lobes affect each other and their borders relate to the appearance of other structures, such as vessels, airways, and the pleural wall. We argue that such structural relationships play a critical role in the accurate delineation of pulmonary lobes when the lungs are affected by diseases such as COVID-19 or COPD. In this paper, we propose a relational approach (RTSU-Net) that leverages structured relationships by introducing a novel non-local neural network module. The proposed module learns both visual and geometric relationships among all convolution features to produce self-attention weights. With a limited amount of training data available from COVID-19 subjects, we initially train and validate RTSU-Net on a cohort of 5000 subjects from the COPDGene study (4000 for training and 1000 for evaluation). Using models pre-trained on COPDGene, we apply transfer learning to retrain and evaluate RTSU-Net on 470 COVID-19 suspects (370 for retraining and 100 for evaluation). Experimental results show that RTSU-Net outperforms three baselines and performs robustly on cases with severe lung infection due to COVID-19.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.
The overview of our lobe segmentation framework with a cascade of two CNNs. At each stage, a CNN (RU-Net) uses the proposed non-local module to capture the structured relationships between objects and object parts. The output from the RU-Net I is concatenated with the cropped 3D patches as the input for RU-Net II.
Fig. 2.
Fig. 2.
Box and whisker plots of IOU per-lobe for different methods on the COPD data set (top) and the COVID-19 data set (bottom).
Fig. 3.
Fig. 3.
Effective Receptive Field (ERF) before the non-local module (2nd row) and after (3rd row) by running forward pass for the first RU-Net on a CT scan from the COVID-19 test set. The green area indicates non-zero gradients (with respect to the input scan) of a feature at a location in the input scan corresponding to the red square (1st row).
Fig. 4.
Fig. 4.
The Self-Attention weights (2nd row) from the proposed non-local module for the feature whose location is shown using the green spot in the original input scan (1st row). We use color map jet [39] for this plot. Two scans from the COVID-19 test set are shown. (a) demonstrates mostly the feature dependencies within the lobe in the clear lung. (b) indicates long-range dependencies are required when the target lobe is affected by disease.
Fig. 5.
Fig. 5.
Qualitative comparison of segmentation results for six representative test cases. The left three columns show COVID-19 cases, the right three columns show COPD cases. From top to bottom: input image, 3DU-Net baseline, PDV-Net, FRV-Net, the proposed RTSU-Net, and the segmentation reference. formula image right upper, formula image right middle, formula image right lower, formula image left upper, formula image left lower lobes.
See this image and copyright information in PMC

Update of

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Prokop M, van Everdingen W, van Rees Vellinga T, Quarles van Ufford J, Stöger L, Beenen L, Geurts B, Gietema H, Krdzalic J, Schaefer-Prokop C, van Ginneken B, Brink M, and The “COVID-19 Standardized Reporting” Working Group of the Dutch Radiological Society, “CO-RADS – a categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation,” Radiology, p. 201473 , 2020. [Online]. Available: https://pubs.rsna.org/doi/abs/10.1148/radiol.2020201473 - DOI - PMC - PubMed
    1. Kubo M, Niki N, Nakagawa S, Eguchi K, Kaneko M, Moriyama N, Omatsu H, Kakinuma R, and Yamaguchi N, “Extraction algorithm of pulmonary fissures from thin-section CT images based on linear feature detector method,” IEEE Trans Nucl Sci, vol. 46, no. 6, pp. 2128–2133, 1999.
    1. Kubo M, Niki N, Eguchi K, Kaneko M, Kusumoto M, Moriyama N, Omatsu H, Kakinuma R, Nishiyama H, Mori K et al., “Extraction of pulmonary fissures from thin-section CT images using calculation of surface-curvatures and morphology filters,” in ICIP, vol. 2, 2000, pp. 637–640.
    1. Wang J, Betke M, and Ko JP, “Pulmonary fissure segmentation on CT,” Med Image Anal, vol. 10, no. 4, pp. 530–547, 2006. - PMC - PubMed
    1. Pu J, Leader JK, Zheng B, Knollmann F, Fuhrman C, Sciurba FC, and Gur D, “A computational geometry approach to automated pulmonary fissure segmentation in CT examinations,” IEEE Trans Med Imaging, vol. 28, no. 5, pp. 710–719, 2008. - PMC - PubMed

Publication types