Curriculum learning applied to the combined algorithm selection and hyperparameter optimization problem

• Ano de defesa: 2021
• Autor(a) principal: Silva, Lucas Nildaimon dos Santos
• Orientador(a): Silva, Diego Furtado
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Universidade Federal de São Carlos; Câmpus São Carlos
• Programa de Pós-Graduação: Programa de Pós-Graduação em Ciência da Computação - PPGCC
• Palavras-chave em Português: Aprendizagem curricular,; Aprendizagem de máquina
• Palavras-chave em Inglês: AutoML; Curriculum learning; Machine learning
• Área do conhecimento CNPq: CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO::METODOLOGIA E TECNICAS DA COMPUTACAO
• Link de acesso: https://repositorio.ufscar.br/handle/20.500.14289/14791

AutoML has the goal to find the best Machine Learning (ML) pipeline in a complex and high dimensional search space by evaluating multiple algorithm configurations. Training multiple ML algorithms is time costly, and as AutoML tools usually obey a time constraint, the exploration of the search space may find sub-optimal results. In this work, we explore the application of curriculum learning techniques to overcome this limitation. Curriculum learning and anti-curriculum learning are strategies for ordering examples during model training based on their difficulty. These have shown to improve model performance and accelerate the training process on previous empirical investigations using optimization-based models. We apply and compare curriculum strategies on two optimizers of an AutoML system to accelerate the search space exploration and find good performing machine learning pipelines with efficiency. The results indicate that AutoML can benefit from a curriculum strategy. In most of the evaluated scenarios, the curriculum strategies led the AutoML algorithm to better classification results.