COVID-19 Detection from Chest X-Ray Images Using CNNs Models : Further Evidence from Deep Transfer Learning

Methods: We collect a relatively large COVID-19 dataset comparing with previous studies that contain 309 real COVID-19 chest x-ray images. We also prepare 2,000 chest x-ray images of pneumonia cases and 1,000 images of healthy chest cases. Deep Transfer Learning is used to detect abnormalities in our image dataset. We fine-tune three, pre-trained convolutional neural networks (CNNs) models on a training dataset: DenseNet 121, NASNetLarge, and NASNetMobile.


Introduction
On December 2019, the novel coronavirus (COVID-19) has started to spread in China, then in other multiple countries around the world. [1][2][3] The early automatic diagnosis of this disease may be very beneficial for reducing its spread. [4] Deep Learning is one artificial intelligence method that can be helpful in detecting COVID-19 infections from medical images such as X-ray images, specifically when we have a small image dataset. [2,4,5] Earlier studies have used Deep Learning for the detection of COVID-19 from chest X-ray images. Minaee et al. prepared a dataset of 5071 chest X-ray images including 71 COVID-19 images and 5000 non COVID-19 images. [2] They selected 40 COVID-19 images and 3000 non COVID-19 to include in the test set, as well as 31 COVID-19 (496 after augmentation) and 2000 non COVID-19 images for the training set. They trained four popular Deep Learning models, including ResNet18, ResNet50, DenseNet-121, and SqueezeNet, to detect COVID-19 infections. The best performing model achieved a sensitivity rate of 97.5%, and 95% of specificity. [2] Apostolopoulos and Mpesiana evaluated the performance of different state-of-the-art CNNs architectures on two datasets. [4] The first one contains 1427 chest X-ray images including 224 images with COVID-19 infection, 700 images with confirmed common bacterial pneumonia, and 504 images of healthy cases. The second dataset includes 224 images with COVID-19 infection, 714 images with confirmed bacterial and viral pneumonia, as well as 504 images of healthy cases. [4] According to their results, the best achieved accuracy, sensitivity, and specificity is 96.78%, 98.66%, and 96.46% respectively. [4] Ozturk et al. proposed the DarkCovidNet Model for the identification of COVID-19 from chest X-ray images. [6] They evaluate their model on a datasets including 127 chest X-ray images of COVID-19 infection as well ULJRI COVID-19 Detection from Chest X-Ray Images Using CNNs Models: Further Evidence from Deep Transfer Learning as 500 healthy and 500 pneumonia chest X-ray images. [6] Their model produced a classification accuracy of 98.08% for binary classes (COVID-19 vs. Healthy) and 87.02% for multi-class cases (COVID-19 vs. Healthy vs. Pneumonia). [6] In this study we use Deep Transfer Leaning to identify COVID-19 infection on a relatively large COVID-19 dataset (without augmentation) comparing with previous studies, in order to get more reliable diagnostic performance, based on two evaluation metrics; sensitivity and specificity.

Methods Datasets
A total of 309 COVID-19 chest X-ray images (excluding lateral images) are collected; 236 COVID-19 images are obtained from the datasets of Cohen et al [7,8] and 73 other COVID-19 images are obtained from Kaggle dataset. [9] We also prepare 2,000 pneumonia and 1,000 healthy chest X-ray images, collected from the dataset of Kermany et al. [10] All images are resized to 224 x 224 pixels. We split our datasets into training set (80%) and testing set (20%), as it is descript in Table 1 above. Data augmentation techniques are not used in this study.

Models
The Deep Transfer Learning method is used in this study, because the samples in our datasets are small and not sufficient to train a CNN model from onset. [2,4,5] Transfer learning consists of extracting features learned on one problem, and using them on a new, similar problem. [11] In our case, we extract features learned on the ImageNet dataset, a large dataset including of 1.4M images and 1000 classes [5], and leveraging them in the detection of COVID-19 infection.
We fine-tune three pre-trained deep CNNs models on the train dataset: DenseNet 121 [12], NASNetLarge, and NAS-NetMobile. [1] Firstly, we instantiate the proposed models pre-loaded with weights trained on ImageNet datasets. [5] We do not include the classification layers at the top, to make the models ideal for feature extraction. [5] Then, we construct a new classifier and add it on top of the base models. The new classifier consists of a pooling layer and a dense fully connected layer. Finally, we freeze the convolutional base before training our models.
The training is done for 10 epochs, with a batch size of 16. We use ADAM optimization with a learning rate of 0.001. We train our models with categorical cross entropy. The implementation is conducted in Python 3 using Keras [11] and Tensorflow [5].
We use sensitivity and specificity metrics to evaluate our models' performance, since the whole dataset is imbalanced. [2] These evaluation metrics are defined as follows [2]:  Where: TP represents images correctly predicted as COVID-19, TN represents images correctly predicted as Non-COVID-19. Table 2 shows the achieved sensitivity and specificity rates by the proposed models. NASNetLarge and NASNetMobile reach a sensitivity rate of 100%, while DenseNet 121 outperforms the two other models in term of specificity rate. Table 3 shows the confusion matrices of DenseNet 121, NASNetLarge and NASNetMobile models, respectively. We can see from this table that our models confuse in the identification between COVID-19 and pneumonia in some cases.

Discussion
In this study, we revisited the detection of COVID-19 form chest X-ray images using Deep Learning. In order to get a more reliable diagnostic performance, we used a relatively large COVID-19 dataset of real chest X -ray images (without augmentation). Our datasets contains 309 chest X-ray images, 2000 pneumonia and 1000 healthy chest X-ray images. The datasets were split into a training set (80%) and a test set (20%). We fine-tuned three pre-trained deep CNNs models: DenseNet 121, NASNetLarge, and NASNetMobile.
We evaluated the performance of our models on the test dataset based on two evaluation metrics: sensitivity, specificity. The proposed models show a good COVID-19 diagnostic performance, where they achieve an average sensitivity rate of approximately 99.45 % and an average specificity rate of approximately 99.5 %. From the confusion matrices, we see that these models confuse in the detection between COVID-19 and pneumonia in some cases.
Since the whole dataset is imbalanced, more metrics are needed to evaluate the performance of our models, such as, the ROC curves and Precision-Recall curves. The used set of COVID-19 X-ray images in this study is limited. Thereby, a larger COVID-19 image dataset could lead to a more reliable diagnosis. [2] The automatic diagnosis of COVID-19 using artificial intelligence could be helpful, but should not replace clinical diagnosis.  Table 3. Confusion matrices of the proposed models.