Evaluating an automated procedure of machine learning parameter tuning for software effort estimation
Saved in:
| Author: | |
|---|---|
| Format: | tesis de maestría |
| Publication Date: | 2021 |
| Description: | Software effort estimation requires accurate prediction models. Machine learning algorithms have been used to create more accurate estimation models. However, these algorithms are sensitive to factors such as the choice of hyper-parameters. To reduce this sensitivity, automated approaches for hyper-parameter tuning have been recently investigated. There is a need for further research on the effectiveness of such approaches in the context of software effort estimation. These evaluations could help understand which hyper-parameter settings can be adjusted to improve model accuracy, and in which specific contexts tuning can benefit model performance. The goal of this work is to develop an automated procedure for machine learning hyper-parameter tuning in the context of software effort estimation. The automated procedure builds and evaluates software effort estimation models to determine the most accurate evaluation schemes. The methodology followed in this work consists of first performing a systematic mapping study to characterize existing hyper-parameter tuning approaches in software effort estimation, developing the procedure to automate the evaluation of hyper-parameter tuning, and conducting controlled quasi experiments to evaluate the automated procedure. From the systematic literature mapping we discovered that effort estimation literature has favored the use of grid search. The results we obtained in our quasi experiments demonstrated that fast, less exhaustive tuners were viable in place of grid search. These results indicate that randomly evaluating 60 hyper-parameters can be as good as grid search, and that multiple state-of-the-art tuners were only more effective than this random search in 6% of the evaluated dataset-model combinations. We endorse random search, genetic algorithms, flash, differential evolution, and tabu and harmony search as effective tuners. |
| Country: | Kérwá |
| Institution: | Universidad de Costa Rica |
| Repositorio: | Kérwá |
| Language: | Inglés |
| OAI Identifier: | oai:kerwa.ucr.ac.cr:10669/84672 |
| Online Access: | https://hdl.handle.net/10669/84672 |
| Keyword: | DESARROLLO DE PROGRAMAS PARA COMPUTADORA APRENDIZAJE POR MAQUINAS |