Мультимодальная ансамблевая нейросетевая система обнаружения рака кожи на гетерогенных дерматологических данных

Ulyana A. Lyakhova; Pavel A. Lyakhov

doi:10.21638/spbu10.2024.208

Authors

Ulyana A. Lyakhova North-Caucasus Federal University, 1, Pushkinа ul., Stavropol, 355017, Russian Federation https://orcid.org/0000-0002-2949-7036
Pavel A. Lyakhov North-Caucasus Federal University, 1, Pushkinа ul., Stavropol, 355017, Russian Federation https://orcid.org/0000-0003-0487-4779

DOI:

https://doi.org/10.21638/spbu10.2024.208

Abstract

Today, skin cancer is one of the leading causes of death in the world. Diagnosing skin cancer early is critical to increasing potential survival. Therefore, it is relevant to develop highprecision intelligent auxiliary diagnostic systems for detecting skin cancer in the early stages. Ensemble learning is one of the current and promising methods for increasing the accuracy of intelligent classification systems by reducing the dispersion and variability of predictions of individual components of the overall system. The work proposes an ensemble intelligent system for analyzing heterogeneous dermatological data based on multimodal neural networks. The accuracy of the developed ensemble system was 85.92 %, which is 1.85 percentage points higher than the average accuracy of individual multimodal architectures for classifying heterogeneous dermatological data. The developed system can be used as a high-precision auxiliary diagnostic tool to help make a medical decision, which will increase the chance of early detection of pigmented oncological pathologies.

Keywords:

multimodal neural network, ensemble neural network, machine learning, heterogeneous data, dermatological images, pigmented skin lesions, skin cancer, melanoma

Downloads

Download data is not yet available.

References

Литература

Apalla Z., Lallas A., Sotiriou E., Lazaridou E., Ioannides D. Epidemiological trends in skin cancer // Dermatol. Pract. Concept. 1885. Vol. 7. Iss. 2. P. 1.

Sinz C., Tschandl P., Rosendahl C., Akay B. N., Argenziano G., Blum A., Kittler H. Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin // Journal of Acad. Dermatol. 2017. Vol. 77. Iss. 6. P. 1100–1109.

Lodha S., Saggar S., Celebi J. T., Silvers D. N. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting // Journal of Cutan Pathol. 2008. Vol. 35. Iss. 4. P. 349–352.

Haggenmüller S., Maron R. C., Hekler A., Utikal J. S., Barata C., Barnhill R. L., Brinker T. J. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts // Eur. Journal of Cancer. 2021. Vol. 156. P. 202–216.

Polikar R., Zhang C., Ma Y. Ensemble Learning // Ensemble Machine Learning. 2012. P. 1–34.

Qureshi A. S., Roos T. Transfer learning with ensembles of deep neural networks for skin cancer detection in imbalanced data sets // Neural Process Lett. 2023. Vol. 55. Iss. 4. P. 4461–4479.

Raza R., Zulfiqar F., Tariq S., Anwar G. B., Sargano A. B., Habib Z. Melanoma classification from dermoscopy images using ensemble of convolutional neural networks // Mathematics. 2021. Vol. 10. Iss. 1. P. 26–43.

Kausar N., Hameed A., Sattar M., Ashraf R., Imran A. S., Abidin M. Z. U., Ali A. Multiclass skin cancer classification using ensemble of fine-tuned deep learning models // Applied Sciences. 2021. Vol. 11. Iss. 22. P. 10593–10608.

Lu Y., Zhang L., Wang B., Yang J. Feature ensemble learning based on sparse autoencoders for image classification // Proceedings of the International Joint Conference on Neural Networks. 2014. P. 1739–1745.

Tang E. K., Suganthan P. N., Yao X. An analysis of diversity measures // Machine Learning. 2006. Vol. 65. Iss. 1. P. 247–271.

Baltruvsaitis T., Ahuja C., Morency L. P. Multimodal machine learning // IEEE Trans. Pattern Anal. Mach. Intell. 2019. Vol. 41. Iss. 2. P. 423–443.

Liu K., Li Y., Xu N., Natarajan P. Learn to combine modalities in multimodal deep learning // arXiv preprint. arXiv:1805.11730. 2018.

Kurtansky N. R., Dusza S. W., Halpern A. C., Hartman R. I., Geller A. C., Marghoob A. A., Marchetti M. A. An epidemiologic analysis of melanoma overdiagnosis in the United States, 1975–2017 // Journal of Investigative Dermatology. 2022. Vol. 142. Iss. 7. P. 1804–1811.

Höhn J., Hekler A., Krieghoff-Henning E., Kather J. N., Utikal J. S., Meier F., Brinker T. J. Integrating patient data into skin cancer classification using convolutional neural networks: systematic review // Journal of Medical Internet Research. 2021. Vol. 23. Iss. 7. P. 20708–20723.

Sriwong K., Bunrit S., Kerdprasop K., Kerdprasop N. Dermatological classification using deep learning of skin image and patient background knowledge // International Journal of Machine Learning and Computing. 2019. Vol. 9. Iss. 6. P. 862–867.

Siegel J. A., Korgavkar K., Weinstock M. A. Current perspective on actinic keratosis: a review // British Journal of Dermatology. 2017. Vol. 177. Iss. 2. P. 350–358.

Lyakhov P. A., Lyakhova U. A., Nagornov N. N. System for the recognizing of pigmented skin lesions with fusion and analysis of heterogeneous data based on a multimodal neural network // Cancers. 2022. Vol. 14. P. 1819–1836.

Chicco D., Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation // BMC Genomics. 2020. Vol. 21. Iss. 1. P. 1–13.

Harangi B., Baran A., Hajdu A. Classification of skin lesions using an ensemble of deep neural networks // Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2018. P. 2575–2578.

Akter M. S., Shahriar H., Sneha S., Cuzzocrea A. Multi-class skin cancer classification architecture based on deep convolutional neural network // 2022 IEEE International Conference on Big Data. Proceedings. 2022. P. 5404–5413.

Keerthana D., Venugopal V., Nath M. K., Mishra M. Hybrid convolutional neural networks with SVM classifier for classification of skin cancer // Biomedical Engineering Advances. 2023. Vol. 5. P. 100069–100103.

References

Apalla Z., Lallas A., Sotiriou E., Lazaridou E., Ioannides D. Epidemiological trends in skin cancer. Dermatol. Pract. Concept., 1885, vol. 7, iss. 2, pp. 1.

Sinz C., Tschandl P., Rosendahl C., Akay B. N., Argenziano G., Blum A., Kittler H. Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin. Journal of Acad. Dermatol., 2017, vol. 77, iss. 6, pp. 1100–1109.

Lodha S., Saggar S., Celebi J. T., Silvers D. N. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting. Journal of Cutan Pathol., 2008, vol. 35, iss. 4, pp. 349–352.

Haggenmüller S., Maron R. C., Hekler A., Utikal J. S., Barata C., Barnhill R. L., Brinker T. J. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur. Journal of Cancer., 2021, vol. 156, pp. 202–216.

Polikar R., Zhang C., Ma Y. Ensemble Learning. Ensemble Machine Learning, 2012, pp. 1–34.

Qureshi A. S., Roos T. Transfer learning with ensembles of deep neural networks for skin cancer detection in imbalanced data sets. Neural Process Lett., 2023, vol. 55, iss. 4, pp. 4461–4479.

Raza R., Zulfiqar F., Tariq S., Anwar G. B., Sargano A. B., Habib Z. Melanoma classification from dermoscopy images using ensemble of convolutional neural networks. Mathematics, 2021, vol. 10, iss. 1, pp. 26–43.

Kausar N., Hameed A., Sattar M., Ashraf R., Imran A. S., Abidin M. Z. U., Ali A. Multiclass skin cancer classification using ensemble of fine-tuned deep learning models. Applied Sciences, 2021, vol. 11, iss. 22, pp. 10593–10608.

Lu Y., Zhang L., Wang B., Yang J. Feature ensemble learning based on sparse autoencoders for image classification. Proceedings of the International Joint Conference on Neural Networks, 2014, pp. 1739–1745.

Tang E. K., Suganthan P. N., Yao X. An analysis of diversity measures. Machine Learning, 2006, vol. 65, iss. 1, pp. 247–271.

Baltruvsaitis T., Ahuja C., Morency L. P. Multimodal machine learning. IEEE Trans. Pattern Anal. Mach. Intell., 2019, vol. 41, iss. 2, pp. 423–443.

Liu K., Li Y., Xu N., Natarajan P. Learn to combine modalities in multimodal deep learning. arXiv preprint. arXiv:1805.11730, 2018.

Kurtansky N. R., Dusza S. W., Halpern A. C., Hartman R. I., Geller A. C., Marghoob A. A., Marchetti M. A. An epidemiologic analysis of melanoma overdiagnosis in the United States, 1975–2017. Journal of Investigative Dermatology, 2022, vol. 142, iss. 7, pp. 1804–1811.

Höhn J., Hekler A., Krieghoff-Henning E., Kather J. N., Utikal J. S., Meier F., Brinker T. J. Integrating patient data into skin cancer classification using convolutional neural networks: systematic review. Journal of Medical Internet Research, 2021, vol. 23, iss. 7, pp. 20708–20723.

Sriwong K., Bunrit S., Kerdprasop K., Kerdprasop N. Dermatological classification using deep learning of skin image and patient background knowledge. International Journal of Machine Learning and Computing, 2019, vol. 9, iss. 6, pp. 862–867.

Siegel J. A., Korgavkar K., Weinstock M. A. Current perspective on actinic keratosis: a review. British Journal of Dermatology, 2017, vol. 177, iss. 2, pp. 350–358. pagebreak

Lyakhov P. A., Lyakhova U. A., Nagornov N. N. System for the recognizing of pigmented skin lesions with fusion and analysis of heterogeneous data based on a multimodal neural network. Cancers, 2022, vol. 14, pp. 1819–1836.

Chicco D., Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 2020, vol. 21, iss. 1, pp. 1–13.

Harangi B., Baran A., Hajdu A. Classification of skin lesions using an ensemble of deep neural networks. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2018, pp. 2575–2578.

Akter M. S., Shahriar H., Sneha S., Cuzzocrea A. Multi-class skin cancer classification architecture based on deep convolutional neural network. 2022 IEEE International Conference on Big Data. Proceedings, 2022, pp. 5404–5413.

Keerthana D., Venugopal V., Nath M. K., Mishra M. Hybrid convolutional neural networks with SVM classifier for classification of skin cancer. Biomedical Engineering Advances, 2023, vol. 5, pp. 100069–100103.