Skin Tone Classification of Human Faces using the MST Scale

• Ano de defesa: 2025
• Autor(a) principal: Matias, Vitor Pereira
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Aprendizado Profundo; Classificação; Classification; Computer vision; Criação de Datasets; Dataset creation; Deep learning; Skin tone; Tons de Pele; Visão Computacional
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-27082025-203858/

Skin tone is a critical attribute for fairness and accountability in computer vision (CV), often approximated by ethnicity. CV algorithms are usually data hungry. However, to perform skin tone classification, existing benchmarks lack diversity. Also, available datasets are limited in size, accessibility and range of skin tone scales (≤le;6 classes). We introduced a dataset with 42,313 facial images from 3,564 individuals, sourced from public datasets (e.g., CASIA-WebFace, CASIA Face-Africa, LFW) and annotated using the 10-class Monk Skin Tone (MST) Scale. We proposed and evaluated two classification pipelines: SkinToneCCV, a classical approach using preprocessing and hand-crafted features, and SkinToneNet, a Convolutional Neural Network (CNN). Evaluation spanned image- and identity-level dataset splits, including generalization toward out-of-domain MST-labelled datasets. We further studied four loss functions: Cross-Entropy, Weighted Cross-Entropy, Ordinal Cross-Entropy (OCE) and Weighted OCE showing that DenseNet121 with OCE offered the best trade-off between accuracy and ordinal consistency. SkinToneNet significantly outperformed SkinToneCCV, with DenseNet121 achieving 93.2% weighted off-by-one accuracy under the identity-level split. These results established a strong MST-based baseline and demonstrated the effectiveness of ordinal-aware losses for skin tone prediction.