Journal of Education and Health Promotion

Performance evaluation of selected machine learning algorithms for COVID‑19 prediction using routine clinical data: With versus Without CT scan features

Document Type : Original Article

Authors

. Mostafa Shanbehzadeh 1
. Hadi Kazemi-Arpanahi 2
. Azam Orooji 3
. Sara Mobarak 4
. Saeed Jelvay 5

1 Assistant Professor of Health Information Management, Department of Health Information Technology, School of Paramedical, Ilam University of Medical Sciences, Ilam, Iran

2 Assistant Professor of Health Information Management, Department of Health Information Technology, Abadan University of Medical Sciences, Abadan, Iran, Assistant Professor of Health Information Management, Student Research Committee, Abadan University of Medical Sciences, Abadan, Iran

3 Assistant Professor of Medical Informatics, School of Medicine, North Khorasan University of Medical Science, North Khorasan, Iran

4 Assistant Professor of Infectious Diseases, School of Medicine, Abadan University of Medical Sciences, Abadan, Iran

5 MSc of Health Information Technology, Department of Student Research Committee, Abadan University of Medical Sciences, Abadan, Iran

Abstract
BACKGROUND: Given coronavirus disease (COVID‑19’s) unknown nature, diagnosis, and treatment
is very complex up to the present time. Thus, it is essential to have a framework for an early prediction
of the disease. In this regard, machines learning (ML) could be crucial to extract concealed patterns
from mining of huge raw datasets then it establishes high‑quality predictive models. At this juncture,
we aimed to apply different ML techniques to develop clinical predictive models and select the best
performance of them.
MATERIALS AND METHODS: The dataset of Ayatollah Talleghani hospital, COVID‑19 focal center
affiliated to Abadan University of Medical Sciences have been taken into consideration. The dataset
used in this study consists of 501 case records with two classes (COVID‑19 and non COVID‑19) and
32 columns for the diagnostic features. ML algorithms such as Naïve Bayesian, Bayesian Net, random
forest (RF), multilayer perceptron, K‑star, C4.5, and support vector machine were developed. Then,
the recital of selected ML models was assessed by the comparison of some performance indices such
as accuracy, sensitivity, specificity, precision, F‑score, and receiver operating characteristic (ROC).
RESULTS: The experimental results indicate that RF algorithm with the accuracy of 92.42%, specificity
of 75.70%, precision of 92.30%, sensitivity of 92.40%, F‑measure of 92.00%, and ROC of 97.15%
has the best capability for COVID‑19 diagnosis and screening.
CONCLUSION: The empirical results reveal that RF model yielded higher performance as compared
to other six classification models. It is promising to the implementation of RF model in the health‑care
settings to increase the accuracy and speed of disease diagnosis for primary prevention, screening,
surveillance, and early treatment.

Keywords

Artificial intelligence
computed tomography scan
coronavirus
COVID‑19
data mining
machine learning
random forest
  1. Rao AS, Vazquez JA. Identification of COVID‑19 can be quicker
    through artificial intelligence framework using a mobile phone–
    based survey when cities and towns are under quarantine. Infect
    Control Hosp Epidemiol 2020;41:826‑30.
    2. Chan JF, Yuan S, Kok KH, To KK, Chu H, Yang J, et al. A familial
    cluster of pneumonia associated with the 2019 novel coronavirus
    indicating person‑to‑person transmission: A study of a family
    cluster. Lancet 2020;395:514‑23.
    3. BikdeliB, TalasazAH, RashidiF, Sharif‑KashaniB, FarrokhpourM,
    Bakhshandeh H, et al. Intermediate versus standard‑dose
    prophylactic anticoagulation and statin therapy versus placebo
    in critically‑ill patients with COVID‑19: Rationale and design
    of the INSPIRATION/INSPIRATION‑S studies. Thromb Res
    2020;196:382‑94.
    4. Panahi S, Ashrafi‑Rizi H, Panahi M. Exposure to
    coronavirus (COVID‑19) using narrative and simulated
    experience approaches: A commentary. J Educ Health Promot
    2020;9:135.5. Peeri NC, Shrestha N, Rahman MS, Zaki R, Tan Z, Bibi S, et al.
    The SARS, MERS and novel coronavirus (COVID‑19) epidemics,
    the newest and biggest global health threats: What lessons have
    we learned? Int J Epidemiol 2020;49:717‑26.
    6. Mackenzie JS, Smith DW. COVID‑19‑A novel zoonotic disease:
    A review of the disease, the virus, and public health measures.
    Asia Pac J Public Health 2020;32:145‑53.
    7. Yoo HS, Yoo D. COVID‑19 and veterinarians for one health,
    zoonotic‑and reverse‑zoonotic transmissions. J Vet Sci
    2020;21:e51.
    8. Cascella M, Rajnik M, Cuomo A, Dulebohn SC, Di Napoli R.
    Features, evaluation and treatment coronavirus (COVID‑19). In:
    StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing;
    2020.
    9. Sohrabi C, Alsafi Z, O’Neill N, Khan M, Kerwan A, Al‑Jabir A,
    et al. World Health Organization declares global emergency:
    A review of the 2019 novel coronavirus (COVID‑19). Int J Surg
    2020;76:71‑6.
    10. Shanbehzadeh M, Kazemi‑Arpanahi H, Mazhab‑Jafari K,
    Haghiri H. Coronavirus disease 2019 (COVID‑19) surveillance
    system: Development of COVID‑19 minimum data set and
    interoperable reporting framework. J Educ Health Promot. 2020
    Aug 31;9:203.
    11. Jacobsen KH. Will COVID‑19 generate global preparedness?
    Lancet 2020;395:1013‑4.
    12. Wang P, Zheng X, Li J, Zhu B. Prediction of epidemic trends in
    COVID‑19 with logistic model and machine learning technics.
    Chaos Solitons Fractals 2020;139:110058.
    13. El Zowalaty ME, Järhult JD. From SARS to COVID‑19:
    A previously unknown SARS‑related coronavirus (SARS‑CoV‑2)
    of pandemic potential infecting humans – Call for a one health
    approach. One Health 2020;9:100124.
    14. Torrealba‑Rodriguez O , C o n d e ‑ G u t i é r r e z R A ,
    Hernández‑Javier AL. Modeling and prediction of COVID‑19 in
    Mexico applying mathematical and computational models. Chaos
    Solitons Fractals 2020;138:109946.
    15. Liu Y, Wang Z, Ren J, Tian Y, Zhou M, Zhou T, et al. A COVID‑19
    risk assessment decision support system for general practitioners:
    Design and development study. JMed Internet Res 2020;22:e19786.
    16. ALOM, Md Zahangir, et al. COVID_MTNet: COVID‑19 Detection
    with Multi‑Task Deep Learning Approaches. arXiv preprint arXiv:
    2004.03747, 2020.‏
    17. Bansal A, Padappayil RP, Garg C, Singal A, Gupta M, Klein A.
    Utility of artificial intelligence amidst the COVID 19 pandemic:
    A review. J Med Syst 2020;44:156.
    18. Lai CC, Shih TP, Ko WC, Tang HJ, Hsueh PR. Severe acute
    respiratory syndrome coronavirus 2 (SARS‑CoV‑2) and
    coronavirus disease‑2019 (COVID‑19): The epidemic and the
    challenges. Int J Antimicrob Agents 2020;55:105924.
    19. Hussain A, Bhowmik B, do Vale Moreira NC. COVID‑19 and
    diabetes: Knowledge in progress. Diabetes Res Clin Pract
    2020;162:108142.
    20. Moujaess E, Kourie HR, Ghosn M. Cancer patients and research
    during COVID‑19 pandemic: A systematic review of current
    evidence. Crit Rev Oncol Hematol 2020;150:102972.
    21. Thabtah F, Peebles D. A new machine learning model based
    on induction of rules for autism detection. Health Informatics J
    2020;26:264‑86.
    22. Gunčar G, Kukar M, Notar M, Brvar M, Černelč P, Notar M, et al.
    An application of machine learning to haematological diagnosis.
    Sci Rep 2018;8:1‑12.
    23. Bryce C, Ring P, Ashby S, Wardman JK. Resilience in the face of
    uncertainty: Early lessons from the COVID‑19 pandemic. JRisk Res
    2020; 23 (7): 140‑8. [Doi: https://doi.org/10.1080/13669877.2020
    0.1756379].
    24. Chater N. Facing up to the uncertainties of COVID‑19. Nat Hum
    Behav 2020;4:439.
    25. Mei X, Lee HC, Diao KY, Huang M, Lin B, Liu C, et al.
    Artificial intelligence–enabled rapid diagnosis of patients with
    COVID‑19. Nat Med. 2020 Aug; 26 (8):1224‑1228. [Doi: 10.1038/
    s41591‑020‑0931‑3].
    26. Wang Y, Song W, Zhao Z, Chen P, Liu J, Li C. The impacts of
    viral inactivating methods on quantitative RT‑PCR for COVID‑19.
    Virus Res 2020;285:197988.
    27. Brogna B, Bignardi E, Salvatore P, Alberigo M, Brogna C,
    Megliola A, et al. Unusual presentations of COVID‑19 pneumonia
    on CT scans with spontaneous pneumomediastinum and
    loculated pneumothorax: A report of two cases and a review of
    the literature. Heart Lung 2020; 49:864‑68. [Doi: 10.1016/j. hrtlng.
    2020.06.005].
    28. Chen J, Peng S, Zhang B, Liu Z, Liu L, Zhang W. An uncommon
    manifestation of COVID‑19 pneumonia on CT scan with small
    cavities in the lungs: A case report. Medicine (Baltimore)
    2020;99:e21240.
    29. Gündüz Y, Öztürk MH, Tomak Y. The usual course of thorax
    CT findings of COVID‑19 infection and when to perform control
    thorax CT scan. Turk J Med Sci 2020;50:684‑6.
    30. Hassanien AE, Salama A, Darwsih A. Artificial intelligence
    approach to predict the COVID‑19 patient’s recovery. No 3223
    Easy Chair; 2020.
    31. Jin C, Chen W, Cao Y, Xu Z, Tan Z, Zhang X, et al. Development
    and evaluation of an artificial intelligence system for COVID‑19
    diagnosis. Nat Commun 2020;11:5088.
    32. Mei X, Lee HC, Diao KY, Huang M, Lin B, Liu C, et al. Artificial
    intelligence–enabled rapid diagnosis of patients with COVID‑19.
    Nat Med 2020;26:1224‑8.
    33. VaishyaR, JavaidM, KhanIH, HaleemA. Artificial intelligence (AI)
    applications for COVID‑19 pandemic. Diabetes Metab Syndr
    2020;14:337‑9.
    34. Wong ZSY, Zhou J, Zhang Q. Artificial Intelligence for infectious
    disease big data analytics. Infect Dis Health 2019;24:44‑8.
    35. Wu CC, Yeh WC, Hsu WD, Islam MM, Nguyen PAA, Poly TN,
    et al. Prediction of fatty liver disease using machine learning
    algorithms. Comput Methods Programs Biomed 2019;170:23‑9.
    36. Vinod DN, Prabaharan SRS. Data science and the role of artificial
    intelligence in achieving the fast diagnosis of COVID‑19. Chaos
    Solitons Fractals 2020;140:110182.
    37. Albahri AS, Hamid RA, Alwan JK, Al‑Qays ZT, Zaidan AA,
    Zaidan BB, et al. Role of biological data mining and machine
    learning techniques in detecting and diagnosing the novel
    coronavirus (COVID‑19): A systematic review. J Med Syst
    2020;44:122.
    38. Sharma S. Drawing insights from COVID‑19‑infected patients
    using CT scan images and machine learning techniques: A study
    on 200 patients. Environ Sci Pollut Res Int 2020;27:37155‑63.
    39. Mantas J. Setting up an Easy‑to‑use machine learning pipeline for
    medical decision support: A case study for COVID‑19 diagnosis
    based on deep learning with CT scans. Importance Health Inform
    Public Health Pandemic 2020;272:13.
    40. Li WT, Ma J, Shende N, Castaneda G, Chakladar J, Tsai JC, et al.
    Using machine learning of clinical data to diagnose COVID‑19:
    A systematic review and meta‑analysis. BMC Med Inform Decis
    Mak 2020;20:247.
    41. Agbehadji IE, Awuzie BO, Ngowi AB, Millham RC. Review
    of big data analytics, artificial intelligence and nature‑inspired
    computing models towards accurate detection of COVID‑19
    pandemic cases and contact tracing. Int J Environ Res Public
    Health 2020;17: 5330.
    42. Shanbehzadeh M, Kazemi‑Arpanahi H. Development of minimal
    basic data set to report COVID‑19. Med J Islam Repub Iran
    2020;34:754‑63.
    43. Chapman BP, Weiss A, Duberstein PR. Statistical learning theory
    for high dimensional prediction: Application to criterion‑keyed
    scale development. Psychol Methods 2016;21:603‑20.
  2. 44. Chao CM, Yu YW, Cheng BW, Kuo YL. Construction the model
    on the breast cancer survival analysis use support vector machine,
    logistic regression and decision tree. J Med Syst 2014;38:106.
    45. Wang GL, Li YF, Bi DX. Support vector networks in adaptive
    friction compensation. IEEE Trans Neural Netw 2007;18:1209‑19.
    46. Abdar M, Kalhori SR, Sutikno T, Subroto IM, Arji G. Comparing
    performance of data mining algorithms in prediction heart
    diseases. Int J Electr Comput Eng (2088‑8708) 2015;5:1569‑76. [Doi:
    10.11591/ijece. v5i6.pp1569‑1576].
    47. Kaur G, Chhabra A. Improved J48 classification algorithm for
    the prediction of diabetes. Int J Comput Appl 2014;98:41. [Doi:
    10.1186/s12911‑019‑0790‑3].
    48. Wu X, Kumar V, Quinlan JR, Ghosh J, Yang Q, Motoda H, et al.
    Top 10 algorithms in data mining. Knowl Inf Syst 2008;14:1‑37.
    49. Ozçift A. Random forests ensemble classifier trained with data
    resampling strategy to improve cardiac arrhythmia diagnosis.
    Comput Biol Med 2011;41:265‑71.
    50. Liaw A, Wiener M. Classification and regression by random forest.
    R News. 2002;2:18‑22.
    51. Amirkhani H, Rahmati M, Lucas PJF, Hommersom A. Exploiting
    experts’ knowledge for structure learning of Bayesian networks.
    IEEE Trans Pattern Anal Mach Intell 2017;39:2154‑70.
    52. Zhang S, Tjortjis C, Zeng X, Qiao H, Buchan I, Keane J. Comparing
    data mining methods with logistic regression in childhood obesity
    prediction. Inf Syst Front 2009;11:449‑60.
    53. Zhang Q, Deng D, Dai W, Li J, Jin X. Optimization of culture
    conditions for differentiation of melon based on artificial neural
    network and genetic algorithm. Sci Rep 2020;10:3524.
    54. Cho YB, Farrokhkish M, Norrlinger B, Heaton R, Jaffray D,
    Islam M. An artificial neural network to model response of a
    radiotherapy beam monitoring system. Med Phys 2020;47:1983‑94.
    55. Baitharu TR, Pani SK. Analysis of data mining techniques for
    healthcare decision support system using liver disorder dataset.
    Procedia Comput Sci 2016;85:862‑70.
    56. Maliha SK, Islam T, Ghosh SK, Ahmed H, Mollick MR, Ema RR.
    Prediction of cancer using logistic regression, K‑Star and J48
    algorithm. 2019 4th International Conference on Electrical
    Information and Communication Technology, EICT 2019; 2019.
    57. Wiharto W, Kusnanto H, Herianto H. Intelligence system for
    diagnosis level of coronary heart disease with K‑star algorithm.
    Healthc Inform Res 2016;22:30‑8.
    58. Han J, Pei J, Kamber M. Data mining: Concepts and techniques.
    Amsterdam, Netherlands: Elsevier; 2011.
    59. Saba AI, Elsheikh AH. Forecasting the prevalence of COVID‑19
    outbreak in Egypt using nonlinear autoregressive artificial neural
    networks. Process Saf Environ Prot 2020;141:1‑8.
    60. Chretien JP, George D, Shaman J, Chitale RA, McKenzie FE.
    Influenza forecasting in human populations: A scoping review.
    PLoS One 2014;9:e94130.
    61. Afshar S, Afshar S, Warden E, Manochehri H, Saidijam M.
    Application of Artificial Neural Network in miRNA Biomarker
    Selection and Precise Diagnosis of Colorectal Cancer Iran Biomed
    J 2019;23:175‑83.
    62. Lawson AB. Statistical methods in spatial epidemiology.
    Hoboken, New Jersey: John Wiley & Sons; 2013.
    63. Lalmuanawma S, Hussain J, Chhakchhuak L. Applications
    of machine learning and artificial intelligence for
    Covid‑19 (SARS‑CoV‑2) pandemic: A review. Chaos Solitons
    Fractals 2020;139:110059.
    64. Oliveira BA, Oliveira LC, Sabino EC, Okay TS. SARS‑CoV‑2 and
    the COVID‑19 disease: A mini review on diagnostic methods.
    Rev Inst Med Trop Sao Paulo 2020;62:e44.
    65. Mahmood A, Gajula C, Gajula P. Clinical and diagnostic criteria of
    COVID 19; a study of 4659 patients evaluating diagnostic testing
    and establishing an algorithm. J Med Surg Sci 2020;2:2.
    66. Rehm GB, Woo SH, Chen XL, Kuhn BT, Cortes‑Puch I,
    Anderson NR, et al. Leveraging IoTs and Machine learning for
    patient diagnosis and ventilation management in the intensive
    care unit. IEEE Pervasive Comput 2020;19:68‑78.
    67. Narin A, Kaya C, Pamuk Z. Automatic detection of coronavirus
    disease (COVID‑19) using X‑ray images and deep convolutional
    neural networks. arXiv preprint arXiv2020;3:200310849.
    68. Yasar H, Ceylan M. A novel comparative study for detection of
    Covid‑19 on CT lung images using texture analysis, machine
    learning, and deep learning methods. Multimed Tools Appl
    2020;79:1‑25.
    69. Moftakhar L, Mozhgan S, Safe MS. Exponentially increasing
    trend of infected patients with COVID‑19 in Iran: A comparison
    of neural network and ARIMA forecasting models. Iran J Public
    Health 2020;49:92‑100.
    70. Alakus TB, Turkoglu I. Comparison of deep learning approaches
    to predict COVID‑19 infection. Chaos Solitons Fractals
    2020;140:110120. [Doi: 10.1371/journal.pone. 0236621].
    71. Sedik A, Iliyasu AM, El‑Rahiem A, Abdel Samea ME,
    Abdel‑Raheem A, Hammad M, et al. Deploying machine and
    deep learning models for efficient data‑augmented detection of
    covid‑19 infections. Viruses 2020;12:769.
Journal of Education and Health Promotion
Volume 11, Issue 7
August 2021
Pages 1-11

Home

Guide for Authors

Submit Manuscript

Reviewers

Contact Us