Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN)
Abstract
:1. Introduction
- We used the VAE network to produce a noise vector that has the domain information.
- We used heavy-tailed student t-distribution to add diversity in the generated medical images.
- We used an auxiliary classifier to push the network to produce images from a specific category.
- To the best of our knowledge, this is the first time that, instead of using random noise, a separate network was trained to obtain domain information and used that informative noise for the generation of medical images.
1.1. Related Work
GAN Applications in Medical Imaging
2. Materials and Methods
2.1. Proposed Method
2.1.1. Variational Autoencoders (VAEs)
2.1.2. GAN with Student T-Distribution
2.1.3. The Loss Function
2.1.4. Auxiliary Classifier Loss Function
2.2. Experiment Settings
3. Results
4. Discussion
4.1. Limitations
4.2. Future Work
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Zhang, J.; Xie, Y.; Xia, Y.; Shen, C. Attention Residual Learning for Skin Lesion Classification. IEEE Trans. Med. Imaging 2019, 38, 2092–2103. [Google Scholar] [CrossRef] [PubMed]
- Trinidad, C.M.; Torres-Cabala, C.A.; Curry, J.L.; Prieto, V.G.; Aung, P.P. Update on eighth edition American Joint Committee on Cancer classification for cutaneous melanoma and overview of potential pitfalls in histological examination of staging parameters. J. Clin. Pathol. 2019, 72, 265–270. [Google Scholar] [CrossRef] [PubMed]
- Kittler, H.; Pehamberger, H.; Wolff, K.; Binder, M. Diagnostic accuracy of dermoscopy. Lancet Oncol. 2002, 3, 159–165. [Google Scholar] [CrossRef]
- Qin, Z.; Liu, Z.; Zhu, P.; Xue, Y. A GAN-based image synthesis method for skin lesion classification. Comput. Methods Programs Biomed. 2020, 195, 105568. [Google Scholar] [CrossRef]
- Sonthalia, S.; Yumeen, S.; Kaliyadan, F. Dermoscopy Overview and Extradiagnostic Applications; StatPearls Publishing: Treasure Island, FL, USA, 2020. [Google Scholar]
- Meng, X.; Chen, J.; Zhang, Z.; Li, K.; Li, J.; Yu, Z.; Zhang, Y. Non-invasive optical methods for melanoma diagnosis. Photodiagnosis Photodyn. Ther. 2021, 34, 102266. [Google Scholar] [CrossRef] [PubMed]
- Khan, M.A.; Sharif, M.; Akram, T.; Damaševičius, R.; Maskeliūnas, R. Skin Lesion Segmentation and Multiclass Classification Using Deep Learning Features and Improved Moth Flame Optimization. Diagnostics 2021, 11, 811. [Google Scholar] [CrossRef]
- Ahmad, B.; Jun, S. Whether the Health Care Practices For the Patients With Comorbidities Have Changed After the Outbreak of COVID-19; Big Data Public Sentiment Analysis. arXiv 2021, arXiv:2104.12559. [Google Scholar]
- Qureshi, M.A.; Aziz, A.; Ahmed, B.; Khalid, A.; Munir, H. Comparative Analysis and Implementation of Efficient Digital Image Watermarking Schemes. Int. J. Comput. Electr. Eng. 2012, 4, 558–561. [Google Scholar] [CrossRef] [Green Version]
- Kassem, M.A.; Hosny, K.M.; Damaševičius, R.; Eltoukhy, M.M. Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review. Diagnostics 2021, 11, 1390. [Google Scholar] [CrossRef]
- Ahmad, B.; Hussain, I. Design and hardware implementation of ball & beam setup. In Proceedings of the 2017 Fifth International Conference on Aerospace Science & Engineering (ICASE), Islamabad, Pakistan, 14–16 November 2017; pp. 1–6. [Google Scholar]
- You, Q.; Sun, J.; Pan, F.; Palade, V.; Ahmad, B. Dmo-qpso: A multi-objective quantum-behaved particle swarm optimization algorithm based on decomposition with diversity control. Mathematics 2021, 9, 1959. [Google Scholar] [CrossRef]
- Tajbakhsh, N.; Shin, J.Y.; Gurudu, S.R.; Hurst, R.T.; Kendall, C.B.; Gotway, M.B.; Liang, J. Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? IEEE Trans. Med. Imaging 2016, 35, 1299–1312. [Google Scholar] [CrossRef] [Green Version]
- Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.W.M.; van Ginneken, B.; Sánchez, C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. Adv. Neural Inf. Process. Syst. 2014, 27, 2672–2680. [Google Scholar] [CrossRef]
- Shahsavari, A.; Ranjbari, S.; Khatibi, T. Proposing a novel Cascade Ensemble Super Resolution Generative Adversarial Network (CESR-GAN) method for the reconstruction of super-resolution skin lesion images. Inform. Med. Unlocked 2021, 24, 100628. [Google Scholar] [CrossRef]
- Shen, T.; Hao, K.; Gou, C.; Wang, F.-Y. Mass Image Synthesis in Mammogram with Contextual Information Based on GANs. Comput. Methods Programs Biomed. 2021, 202, 106019. [Google Scholar] [CrossRef] [PubMed]
- Fernández Blanco, R.; Rosado, P.; Vegas, E.; Reverter, F. Medical image editing in the latent space of Generative Adversarial Networks. Intell. Med. 2021, 5, 100040. [Google Scholar] [CrossRef]
- Zhang, J.; Yu, L.; Chen, D.; Pan, W.; Shi, C.; Niu, Y.; Yao, X.; Xu, X.; Cheng, Y. Dense GAN and multi-layer attention based lesion segmentation method for COVID-19 CT images. Biomed. Signal Process. Control. 2021, 69, 102901. [Google Scholar] [CrossRef]
- Pang, T.; Wong, J.H.D.; Ng, W.L.; Chan, C.S. Semi-supervised GAN-based Radiomics Model for Data Augmentation in Breast Ultrasound Mass Classification. Comput. Methods Programs Biomed. 2021, 203, 106018. [Google Scholar] [CrossRef]
- Radford, A.; Metz, L.; Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. In Proceedings of the 4th International Conference on Learning Representations, ICLR 2016—Conference Track Proceedings, San Juan, PR, USA, 2–4 May 2016. [Google Scholar]
- Callealta Barroso, F.J.; García-Pérez, C.; Prieto-Alaiz, M. Modelling income distribution using the log Student’s t distribution: New evidence for European Union countries. Econ. Model. 2020, 89, 512–522. [Google Scholar] [CrossRef]
- Van Der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2625. [Google Scholar]
- Perarnau, G.; van de Weijer, J.; Raducanu, B.; Álvarez, J.M. Invertible Conditional GANs for image editing. arXiv 2016, arXiv:1611.06355. [Google Scholar]
- Saxena, D.; Cao, J. Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions. ACM Comput. Surv. 2020, 54, 1–42. [Google Scholar] [CrossRef]
- Hays, J.; Efros, A.A. Scene completion using millions of photographs. ACM Trans. Graph. 2007, 26, 4. [Google Scholar] [CrossRef]
- Kingma, D.P.; Welling, M. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations, ICLR 2014—Conference Track Proceedings, Banff, AB, Canada, 14–16 April 2014; pp. 1–14. [Google Scholar]
- Gurumurthy, S.; Sarvadevabhatla, R.K.; Babu, R.V. DeLiGAN: Generative adversarial networks for diverse and limited data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 166–174. [Google Scholar] [CrossRef] [Green Version]
- Ma, Y.; Zhong, G.; Wang, Y.; Liu, W. MetaCGAN: A Novel GAN Model for Generating High Quality and Diversity Images with Few Training Data. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July2020; pp. 1–7. [Google Scholar]
- Van Den Oord, A.; Kalchbrenner, N.; Kavukcuoglu, K. Pixel recurrent neural networks. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 19–24 June 2016; Volume 4, pp. 2611–2620. [Google Scholar]
- Van Den Oord, A.; Kalchbrenner, N.; Vinyals, O.; Espeholt, L.; Graves, A.; Kavukcuoglu, K. Conditional image generation with PixelCNN decoders. Adv. Neural Inf. Process. Syst. 2016, arXiv:1606.05328, 4797–4805. [Google Scholar]
- Liu, Y.; Chen, A.; Shi, H.; Huang, S.; Zheng, W.; Liu, Z.; Zhang, Q.; Yang, X. CT synthesis from MRI using multi-cycle GAN for head-and-neck radiation therapy. Comput. Med. Imaging Graph. 2021, 91, 101953. [Google Scholar] [CrossRef] [PubMed]
- Jiang, M.; Zhi, M.; Wei, L.; Yang, X.; Zhang, J.; Li, Y.; Wang, P.; Huang, J.; Yang, G. FA-GAN: Fused Attentive Generative Adversarial Networks for MRI Image Super-Resolution. Comput. Med. Imaging Graph. 2021, 92, 101969. [Google Scholar] [CrossRef]
- Mahapatra, D.; Bozorgtabar, B.; Hewavitharanage, S.; Garnavi, R. Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Metzler, J.B., Ed.; Springer: Cham, Switzerland, 2017; pp. 382–390. [Google Scholar]
- Shin, H.-C.; Tenenholtz, N.A.; Rogers, J.K.; Schwarz, C.G.; Senjem, M.L.; Gunter, J.L.; Andriole, K.P.; Michalski, M. Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks. In International Workshop on Simulation and Synthesis in Medical Imaging; Springer: Cham, Switzerland, 2018; pp. 1–11. [Google Scholar]
- Han, C.; Rundo, L.; Araki, R.; Nagano, Y.; Furukawa, Y.; Mauri, G.; Nakayama, H.; Hayashi, H. Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection. IEEE Access 2019, 7, 156966–156977. [Google Scholar] [CrossRef]
- Nie, D.; Trullo, R.; Lian, J.; Wang, L.; Petitjean, C.; Ruan, S.; Wang, Q.; Shen, D. Medical Image Synthesis with Deep Convolutional Adversarial Networks. IEEE Trans. Biomed. Eng. 2018, 65, 2720–2730. [Google Scholar] [CrossRef]
- Zhong, G.; Gao, W.; Liu, Y.; Yang, Y.; Wang, D.H.; Huang, K. Generative adversarial networks with decoder–encoder output noises. Neural Netw. 2020, 127, 19–28. [Google Scholar] [CrossRef]
- Tschandl, P.; Rosendahl, C.; Kittler, H. Data descriptor: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 2018, 5, 1–9. [Google Scholar] [CrossRef]
- Wei, R.; Garcia, C.; El-Sayed, A.; Peterson, V.; Mahmood, A. Variations in Variational Autoencoders—A Comparative Evaluation. IEEE Access 2020, 8, 153651–153670. [Google Scholar] [CrossRef]
- Joarder, A. A short review of multivariate t-distribution a short review of multivariate t-distribution. J. Stat. Res. 2014, 40, 59–72. [Google Scholar]
- Sun, J.; Zhong, G.; Chen, Y.; Liu, Y.; Li, T.; Huang, K. Generative adversarial networks with mixture of t-distributions noise for diverse image generation. Neural Netw. 2020, 122, 374–381. [Google Scholar] [CrossRef] [PubMed]
- Ma, D.; Liu, B.; Kang, Z.; Zhou, J.; Zhu, J.; Xu, Z. Two birds with one stone: Transforming and generating facial images with iterative GAN. Neurocomputing 2020, 396, 278–290. [Google Scholar] [CrossRef] [Green Version]
- Brinker, T.J.; Hekler, A.; Enk, A.H.; Klode, J.; Hauschild, A.; Berking, C.; Schilling, B.; Haferkamp, S.; Schadendorf, D.; Fröhling, S.; et al. A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur. J. Cancer 2019, 111, 148–154. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hekler, A.; Utikal, J.S.; Enk, A.H.; Hauschild, A.; Weichenthal, M.; Maron, R.C.; Berking, C.; Haferkamp, S.; Klode, J.; Schadendorf, D.; et al. Superior skin cancer classification by the combination of human and artificial intelligence. Eur. J. Cancer 2019, 120, 114–121. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hartanto, C.A.; Wibowo, A. Development of Mobile Skin Cancer Detection using Faster R-CNN and MobileNet v2 Model. In Proceedings of the 2020 7th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), Semarang, Indonesia, 24–25 September 2020; pp. 58–63. [Google Scholar]
- Harangi, B.; Baran, A.; Hajdu, A. Assisted deep learning framework for multi-class skin lesion classification considering a binary classification support. Biomed. Signal Process. Control 2020, 62, 102041. [Google Scholar] [CrossRef]
Nv | Mel | Bcc | Bkl | Avg. Acc | ||
---|---|---|---|---|---|---|
Without Augmentation | ||||||
GAN [15] | Precision | 0.73 | 0.88 | 0.86 | 0.80 | 81.0% |
Recall | 0.85 | 0.70 | 0.83 | 0.86 | ||
F1-Score | 0.78 | 0.78 | 0.85 | 0.83 | ||
DeLiGAN [28] | Precision | 0.76 | 0.91 | 0.86 | 0.85 | 83.75% |
Recall | 0.87 | 0.73 | 0.86 | 0.89 | ||
F1-Score | 0.81 | 0.81 | 0.86 | 0.87 | ||
TED-GAN (Proposed) | Precision | 0.87 | 0.94 | 0.94 | 0.84 | 89.25% |
Recall | 0.92 | 0.82 | 0.90 | 0.93 | ||
F1-Score | 0.89 | 0.88 | 0.92 | 0.88 | ||
With Augmentation | ||||||
GAN [15] | Precision | 81 | 90 | 90 | 81 | 85.25% |
Recall | 88 | 81 | 85 | 87 | ||
F1-Score | 0.84 | 0.85 | 0.88 | 0.84 | ||
DeLiGAN [28] | Precision | 85 | 94 | 94 | 83 | 89.3% |
Recall | 90 | 83 | 89 | 92 | ||
F1-Score | 0.87 | 0.88 | 0.91 | 0.87 | ||
TED-GAN (Proposed) | Precision | 90 | 94 | 93 | 93 | 92.5% |
Recall | 94 | 89 | 92 | 95 | ||
F1-Score | 0.92 | 0.91 | 0.92 | 0.94 |
Source | Dataset | Method | No. of Classes | Sensitivity (Recall)% | Specificity (Precision)% | F1-Score | Accuracy | |
---|---|---|---|---|---|---|---|---|
2019 | CNN [44] | CNN | 2 | 92.8 | 68.2 | - | - | |
[45] | HAM10000 | Physicians | 5 | 66 | 62 | - | - | |
CNN | 5 | 86.1 | 89.2 | - | - | |||
2020 | [46] | Private | R-CNN | 2 | - | - | - | 86.3 |
[47] | HAM10000 | GoogLeNet Inception-v3 | 7 | 75.57 | - | - | - | |
2021 | [7] | HAM10000 | KELM | 7 | 90.2 | - | - | 90.67 |
(TED-GAN) Proposed | HAM10000 | TED-GAN & CNN | 4 | 89 | 94 | 0.91 | 92.5 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ahmad, B.; Jun, S.; Palade, V.; You, Q.; Mao, L.; Zhongjie, M. Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN). Diagnostics 2021, 11, 2147. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11112147
Ahmad B, Jun S, Palade V, You Q, Mao L, Zhongjie M. Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN). Diagnostics. 2021; 11(11):2147. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11112147
Chicago/Turabian StyleAhmad, Bilal, Sun Jun, Vasile Palade, Qi You, Li Mao, and Mao Zhongjie. 2021. "Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN)" Diagnostics 11, no. 11: 2147. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11112147