Evaluating an automated procedure of machine learning parameter tuning for software effort estimation

 

Guardado en:
书目详细资料
作者: Villalobos Arias, Leonardo
格式: tesis de maestría
Fecha de Publicación:2021
实物特征:Software effort estimation requires accurate prediction models. Machine learning algorithms have been used to create more accurate estimation models. However, these algorithms are sensitive to factors such as the choice of hyper-parameters. To reduce this sensitivity, automated approaches for hyper-parameter tuning have been recently investigated. There is a need for further research on the effectiveness of such approaches in the context of software effort estimation. These evaluations could help understand which hyper-parameter settings can be adjusted to improve model accuracy, and in which specific contexts tuning can benefit model performance. The goal of this work is to develop an automated procedure for machine learning hyper-parameter tuning in the context of software effort estimation. The automated procedure builds and evaluates software effort estimation models to determine the most accurate evaluation schemes. The methodology followed in this work consists of first performing a systematic mapping study to characterize existing hyper-parameter tuning approaches in software effort estimation, developing the procedure to automate the evaluation of hyper-parameter tuning, and conducting controlled quasi experiments to evaluate the automated procedure. From the systematic literature mapping we discovered that effort estimation literature has favored the use of grid search. The results we obtained in our quasi experiments demonstrated that fast, less exhaustive tuners were viable in place of grid search. These results indicate that randomly evaluating 60 hyper-parameters can be as good as grid search, and that multiple state-of-the-art tuners were only more effective than this random search in 6% of the evaluated dataset-model combinations. We endorse random search, genetic algorithms, flash, differential evolution, and tabu and harmony search as effective tuners.
País:Kérwá
机构:Universidad de Costa Rica
Repositorio:Kérwá
语言:Inglés
OAI Identifier:oai:kerwa.ucr.ac.cr:10669/84672
在线阅读:https://hdl.handle.net/10669/84672
Palabra clave:DESARROLLO DE PROGRAMAS PARA COMPUTADORA
APRENDIZAJE POR MAQUINAS