Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Sep 5;9(9):e106381.
doi: 10.1371/journal.pone.0106381. eCollection 2014.

Diagnostic accuracy of computer-aided detection of pulmonary tuberculosis in chest radiographs: a validation study from sub-Saharan Africa

Marianne Breuninger  1 , Bram van Ginneken  2 , Rick H H M Philipsen  2 , Francis Mhimbira  3 , Jerry J Hella  3 , Fred Lwilla  3 , Jan van den Hombergh  4 , Amanda Ross  5 , Levan Jugheli  6 , Dirk Wagner  7 , Klaus Reither  6
Affiliations

Diagnostic accuracy of computer-aided detection of pulmonary tuberculosis in chest radiographs: a validation study from sub-Saharan Africa

Marianne Breuninger et al. PLoS One. .

Abstract

Background: Chest radiography to diagnose and screen for pulmonary tuberculosis has limitations, especially due to inter-reader variability. Automating the interpretation has the potential to overcome this drawback and to deliver objective and reproducible results. The CAD4TB software is a computer-aided detection system that has shown promising preliminary findings. Evaluation studies in different settings are needed to assess diagnostic accuracy and practicability of use.

Methods: CAD4TB was evaluated on chest radiographs of patients with symptoms suggestive of pulmonary tuberculosis enrolled in two cohort studies in Tanzania. All patients were characterized by sputum smear microscopy and culture including subsequent antigen or molecular confirmation of Mycobacterium tuberculosis (M.tb) to determine the reference standard. Chest radiographs were read by the software and two human readers, one expert reader and one clinical officer. The sensitivity and specificity of CAD4TB was depicted using receiver operating characteristic (ROC) curves, the area under the curve calculated and the performance of the software compared to the results of human readers.

Results: Of 861 study participants, 194 (23%) were culture-positive for M.tb. The area under the ROC curve of CAD4TB for the detection of culture-positive pulmonary tuberculosis was 0.84 (95% CI 0.80-0.88). CAD4TB was significantly more accurate for the discrimination of smear-positive cases against non TB patients than for smear-negative cases (p-value<0.01). It differentiated better between TB cases and non TB patients among HIV-negative compared to HIV-positive individuals (p<0.01). CAD4TB significantly outperformed the clinical officer, but did not reach the accuracy of the expert reader (p = 0.02), for a tuberculosis specific reading threshold.

Conclusion: CAD4TB accurately distinguished between the chest radiographs of culture-positive TB cases and controls. Further studies on cost-effectiveness, operational and ethical aspects should determine its place in diagnostic and screening algorithms.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: Delft Imaging Systems (CAD4TBII, http://www.delftimagingsystems.com/tb-solutions/cad4tb) provided a research grant to Bram van Ginneken and Rick HHM Philipsen for the development of CAD4TB. There are no patents, further products in development or marketed products to declare. This does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.

Figures

Figure 1
Figure 1. Flow chart of individuals taking part in the study.
Figure 2
Figure 2. Distribution of CAD scores for patient groups A (s+/c+ M.tb), B (s−/c+ M.tb), C (s±/c+ NTM), D (s−/c− clin.TB) and F (Controls).
Figure 3
Figure 3. ROC analysis for the detection of M.tb culture-positive individuals.
Legend. —–A (s+/c+ M.tb), B (s−/c+ M.tb) vs. F (Controls): Az = 0.84 (0.80–0.88), - - - A (s+/c+ M.tb), B (s−/c+ M.tb) vs. C (s±/c+ NTM), E (EPTB), F (Controls): Az = 0.81 (0.77–0.85), p = 0.28.
Figure 4
Figure 4. ROC analysis for the detection of M.tb culture-positive individuals by smear status.
Legend. —– A (s+/c+ M.tb) vs. F (Controls): Az = 0.90 (0.86–0.93), - - - B (s−/c+ M.tb) vs. F (Controls): Az = 0.67 (0.58–0.75), p<0.01.
Figure 5
Figure 5. ROC analysis for the detection of M.tb culture-positive individuals by HIV Status.
Legend. —– HIV negative. A (s+/c+ M.tb), B (s−/c+ M.tb) vs. F (Controls): Az = 0.89 (0.85–0.94), - - - HIV positive. A (s+/c+ M.tb), B (s−/c+ M.tb) vs. F (Controls): Az = 0.79 (0.72–0.86), p<0.01.
Figure 6
Figure 6. Comparison of automated and human reading.
Legend. Sensitivity and specificity to distinguish group A (s+/c+ M.tb) and B (s−/c+ M.tb) vs. F (Controls). Line and shaded area: ROC curve and 95% CI for CAD4TB. The expert reader is represented by square symbols, the clinical officer by round symbols. The different fill of the symbols indicate different reading thresholds: empty symbols  = ‘any abnormality’, crossed symbols  = ‘TB consistent abnormalities’ and filled symbols  = ‘abnormalities highly suggestive for TB’.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Story A, Aldridge RW, Abubakar I, Stagg HR, Lipman M, et al. (2012) Active case finding for pulmonary tuberculosis using mobile digital chest radiography: an observational study. Int J Tuberc Lung Dis 16: 1461–1467 Available: http://www.ncbi.nlm.nih.gov/pubmed/22981252 - PubMed
    1. Van't Hoog AH, Meme HK, Van Deutekom H, Mithika AM, Olunga C, et al. (2011) High sensitivity of chest radiograph reading by clinical officers in a tuberculosis prevalence survey. Int J Tuberc Lung Dis 15: 1308–1314 Available: http://www.ingentaconnect.com/content/iuatld/ijtld/2011/00000015/0000001... - PubMed
    1. Den Boon S, White NW, Van Lill SWP, Borgdorff MW, Verver S, et al. (2006) An evaluation of symptom and chest radiographic screening in tuberculosis prevalence surveys. Int J Tuberc Lung Dis 10: 876–882. - PubMed
    1. Van Cleeff M, Kivihya-Ndugga L, Meme H, Odhiambo J, Klatser P (2005) The role and performance of chest X-ray for the diagnosis of tuberculosis: a cost-effectiveness analysis in Nairobi, Kenya. BMC Infect Dis 5: 111 Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1326228&tool=p.... Accessed 2012 Nov 7 - PMC - PubMed
    1. Graham S, Das GK, Hidvegi RJ, Hanson R, Kosiuk J, et al. (2002) Chest radiograph abnormalities associated with tuberculosis: reproducibility and yield of active cases. Int J Tuberc Lung Dis 6: 137–142 Available: http://www.ncbi.nlm.nih.gov/pubmed/11931412. Accessed 2012 Oct 2 - PubMed

Publication types

Grants and funding

This study was supported by the European and Developing Countries Clinical Trials Partnership (EDCTP) as part of the project “Evaluation of new and emerging diagnostics for childhood tuberculosis in high burden countries” (TB CHILD) [IP.2009.32040.007]. The development of the CAD4TB software was supported by the EDCTP as part of the project “Evaluation of Multiple Novel and Emerging Technologies for TB Diagnosis in Smear negative and HIV infected Persons in High-burden Countries” (TB NEAT study) [IP.09.32040.009]. Delft Imaging Systems (CAD4TBII, http://www.delftimagingsystems.com/tb-solutions/cad4tb) provided a research grant to Bram van Ginneken and Rick HHM Philipsen for the development of CAD4TB. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

LinkOut - more resources