Hybrid Machine Learning Algorithms in High-Dimensional Settings

• Ano de defesa: 2025
• Autor(a) principal: Galvão, Luciano Ribeiro
• Tipo de documento: Tese
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Algorithmic stability; Asymptotic consistency; Consistência assintótica; Convex penalization; Friedman simulations; Estabilidade algorítmica; Generalização preditiva; Interação não linear; Non-linear interaction; Penalização convexa; Predictive generalization; Simulações de Friedman; Stratified cross-validation; Validação cruzada estratificada
• Link de acesso: https://www.teses.usp.br/teses/disponiveis/11/11134/tde-06012026-111153/

This work proposes and evaluates a hybrid framework for prediction and variable selection in supervised learning settings, combining statistical regularization with non-linear machine learning algorithms. Initially, the pipeline is applied to real-world data from a travel insurance company, consisting of 2,697 customer records and 35 explanatory variables obtained through feature engineering. Variables were selected using penalized regressions (Lasso, Ridge, and Elastic Net), followed by black-box predictive models (Random Forest, XGBoost, H2O GBM, LightGBM, and CatBoost) in a hybrid structure. Stratified cross-validation and random search optimization allowed for a comprehensive comparison of metrics such as AUC, F1-score, precision, and recall. The CatBoost model with Lasso-based feature selection achieved the best performance (AUC = 0.861; F1 = 0.808), highlighting the potential of hybrid models to preserve interpretability while maintaining high predictive power. Complementarily, a theoretical and empirical analysis is developed on the asymptotic consistency and generalization properties of such models in high-dimensional noisy settings, using simulations based on the Friedman equation. A total of 23 models are evaluated (3 regularized, 5 pure black-box, and 15 hybrids) across scenarios with varying sample sizes (n = 50, 100, 200, 500, 1000) and numbers of predictors (p = 10, 50, 100). The assessment metrics include RMSE, Jaccard index, and support recovery rate. Results show that hybrid models achieve a superior balance between accurate prediction, parsimony, and robustness in variable selection, especially as increases. Therefore, this study contributes both with unexplored practical applications in the insurance sector and with novel theoretical advances for the formal validation of hybrid pipelines in complex high-dimensional scenarios.