Implementation of Vision Transformers Model in Website-Based Facial Skin Type Classification
DOI:
https://doi.org/10.37859/coscitech.v6i2.10026
Abstract
Skin type misidentification often leads to inappropriate skincare product selection, which can negatively affect skin health. This study aims to develop a web-based automatic facial skin type classification system using the Vision Transformer (ViT) architecture. The model implemented is ViT Base Patch 16, pre-trained on the ImageNet dataset and fine-tuned using 10,000 facial images evenly distributed across four classes: normal, dry, oily, and combination. The dataset underwent augmentation and normalization during preprocessing. The training results showed an accuracy of 78% on the test data, with the best performance in the combination skin class (F1-score of 0.86) and the lowest in the normal skin class (F1-score of 0.72). The model was integrated into a Flask-based system that enables users to classify their skin type by either uploading an image or capturing it via camera. System testing was conducted using functional testing and API testing via Postman. The results demonstrated that all key features of the system functioned properly, and the API successfully returned classification responses in JSON format. This system can assist users in identifying their skin type and serve as a reference for selecting appropriate skincare ingredients.
Downloads
References
O. Sumant, “Dermatologicals Market Expected to Reach $59.309 Billion by 2030,” Allied Market Research. Accessed: Jan. 23, 2025. [Online]. Available: https://www.alliedmarketresearch.com/press-release/dermatological-drugs-market.html
K. Hidayat, “Industri Kosmetik Diprediksi Tetap Tumbuh di Tahun 2025,” Kontan.co.id. Accessed: Jan. 23, 2025. [Online]. Available: https://industri.kontan.co.id/news/industri-kosmetik-diprediksi-tetap-tumbuh-di-tahun-2025
S. D. Kusumaningrum, “Kajian Pustaka Dalam Penentuan Tipe Dan Permasalahan Kulit Wajah,” Jurnal SNATi, vol. 1, no. 1, p. 17, 2021, [Online]. Available: https://www.google.com/
R. Fadil, “Ini yang Dimaksud dengan Breakout Wajah, Penyebab dan Cara Mengobatinya,” Halodoc. Accessed: Jan. 28, 2025. [Online]. Available: https://www.halodoc.com/artikel/ini-yang-dimaksud-dengan-breakout-wajah-penyebab-dan-cara-mengobatinya?srsltid=AfmBOoqliIH8dsDRY5VyPv4WRJDExWq9eXOwJCPXPFOhjFI00gwN_ydZ
M. A. Arshed, S. Mumtaz, M. Ibrahim, S. Ahmed, M. Tahir, and M. Shafi, “Multi-Class Skin Cancer Classification Using Vision Transformer Networks and Convolutional Neural Network-Based Pre-Trained Models,” Information (Switzerland), vol. 14, no. 7, Jul. 2023, doi: 10.3390/info14070415.
C. Flosdorf, J. Engelker, I. Keller, and N. Mohr, “Skin Cancer Detection utilizing Deep Learning: Classification of Skin Lesion Images using a Vision Transformer,” 2024. [Online]. Available: www.aaai.org
G. Boesch, “Vision Transformers (ViT) in Image Recognition – 2024 Guide,” viso.ai. Accessed: Jan. 28, 2025. [Online]. Available: https://viso.ai/deep-learning/vision-transformer-vit/
D. A. Purboningtyas, “Klasifikasi Jenis Kulit Wajah Menggunakan Metode Vision Transformers,” Universitas Islam Sultan Agung, Semarang, 2024.
Dian Anisa Agustina, “Klasifikasi Citra Jenis Kulit Wajah Dengan Algoritma Convolutional Neural Network (Cnn) Resnet-50,” Jurnal Riset Sistem Informasi, vol. 1, no. 3, pp. 01–07, Jul. 2024, doi: 10.69714/13sbby24.
M. Ath-Thariq and T. N. Suharsono, “Deteksi Penyakit Kulit Serupa Pada Wajah Berbasis Mobile dengan Metode Convolutional Neural Network,” INNOVATIVE: Journal Of Social Science Research, vol. 3, no. 5, pp. 876–887, Oct. 2023, Accessed: May 28, 2025. [Online]. Available: https://j-innovative.org/index.php/Innovative/article/view/4936
A. Munib and F. Wulandari, “Studi Literatur: Efektivitas Model Kooperatif Tipe Course Review Horay Dalam Pembelajaran Ipa Di Sekolah Dasar,” Jurnal Pendidikan Dasar Nusantara, vol. 7, no. 1, pp. 160–172, 2021.
U. Sulung and M. Muspawi, “Memahami Sumber Data Penelitian : Primer, Sekunder, Dan Tersier,” EDU RESEARCH, vol. 5, no. 3, pp. 110–116, Sep. 2024.
Ridwan and N. F. Tungkas, Metode Penelitian, 1st ed. Yogyakarta: Yayasan Sahabat Alam Rafflesia, 2024.
R. O. Waruwu, K. S. Zai, M. M. Bate’e, and J. B. I. J. Gea, “Pengoperasian Sistem Aplikasi E-Arsip Dalam Memaksimalkan Manajemen Operasi Pelayanan Surat Masuk Dan Surat Keluar Berbasis Digital Di Dinas Komunikasi Dan Informatika Kabupaten Nias Utara,” Jurnal EMBA, vol. 12, no. 1, pp. 1044–1051, Jan. 2024.
S. Dissanayake, “Oily, Dry and Normal Skin Types Dataset,” 2024. Accessed: Jun. 30, 2025. [Online]. Available: https://www.kaggle.com/datasets/shakyadissanayake/oily-dry-and-normal-skin-types-dataset
Skin Type, “Skin_Type Computer Vision Project,” 2025. Accessed: Jun. 30, 2025. [Online]. Available: https://universe.roboflow.com/skin-type-jtczu/skin_type-vtn2q
D. M. Pratama, “Implementasi Metode Vision Transformer (Vit) Dalam Klasifikasi Citra Mri Penyakit Alzheimer,” UNIVERSITAS LAMPUNG, Lampung, 2024.
“Source Code for Torchvision,” PyTorch Foundation. Accessed: Jul. 01, 2025. [Online]. Available: https://docs.pytorch.org/vision/main/generated/torchvision.transforms.ToTensor.html
“Introduction to ViT (Vision Transformers): Everything You Need to Know,” Lightly. Accessed: Jul. 01, 2025. [Online]. Available: https://www.lightly.ai/blog/vision-transformers-vit?utm_source=chatgpt.com
M. D. Junas, “Pemodelan Convolutional Vision Transformer Pada Pemrosesan Gambar Spektogram Untuk Deteksi North Atlantic Right Whales Up-Call,” Universitas Islam Negeri Syarif Hidayatullah Jakarta, Jakarta, 2023.
Y. Anis, A. B. Mukti, and A. N. Rosyid, “Penerapan Model Waterfall Dalam Pengembangan Sistem Informasi Aset Destinasi Wisata Berbasis Website,” KLIK: Kajian Ilmiah Informatika dan Komputer, vol. 4, no. 2, pp. 1134–1142, Oct. 2023, doi: 10.30865/klik.v4i2.1287.
M. T. Abdillah et al., “Implementasi Black box Testing dan Usability Testing pada Website Sekolah MI Miftahul Ulum Warugunung Surabaya,” Jurnal Ilmu Komputer dan Desain Komunikasi Visual, vol. 8, no. 1, pp. 234–242, Jul. 2023.
W. A. Firmansyach, U. Hayati, and Y. A. Wijaya, “Analisa Terjadinya Overfitting Dan Underfitting Pada Algoritma Naive Bayes Dan Decision Tree Dengan Teknik Cross Validation,” Jurnal Mahasiswa Teknik Informatika, vol. 7, no. 1, pp. 262–269, Feb. 2023.
