Evaluasi Kritis Random Forest dalam Prediksi Kegagalan CI/CD Pipeline pada Dataset Log Sintetis
DOI:
https://doi.org/10.37859/jf.v16i2.11756
Abstract
Continuous Integration and Continuous Deployment (CI/CD) pipelines are crucial in modern DevOps environments; however, pipeline failures often hinder software delivery. This study aims to implement Machine Learning specifically the Random Forest algorithm to predict the specific stage of failure (Build, Test, or Deploy phases) using execution logs. The research methodology encompasses data preprocessing, the extraction of three key dynamic features (execution duration, CPU usage, and memory consumption), model training using an 80:20 train-test split, and performance evaluation via a confusion matrix. Evaluation based on 9,000 test samples representing 20% of the total 45,000 synthetic failure logs yielded an overall accuracy of 33.22%, with F1-scores of 0.33 for the Build class, 0.35 for Deploy, and 0.32 for Test. These results indicate that the Random Forest model possesses very limited generalization capability because the synthetic dataset lacked naturally robust failure trace patterns, as evidenced by an accuracy level approaching that of random guessing. It is concluded that the use of a synthetic log dataset constitutes the primary limitation and the root cause of the model's generalization failure in this study. Therefore, future research is recommended to utilize actual operational logs from real-world CI/CD platforms, such as GitHub Actions or Jenkins, to enable the model to demonstrate more valid causal relationships.
Downloads
References
J. Smith dan A. Kumar, “The Evolution of DevOps Practices: A Decade in Review,” IEEE Access, vol. 10, hlm. 45120–45135, 2022, doi: 10.1109/ACCESS.2022.3168234.
L. Chen, “Continuous Integration and Deployment in Modern Software Engineering,” Empir. Softw. Eng., vol. 28, no. 3, hlm. 55, 2023, doi: 10.1007/s10664-023-10311-2.
R. Ali dan others, “Analyzing CI/CD Pipelines in Open Source Projects,” Inf. Softw. Technol., vol. 144, hlm. 106798, 2022, doi: 10.1016/j.infsof.2021.106798.
M. Waseem, P. Liang, dan M. Shahin, “DevOps Transformation: A Systematic Literature Review,” IEEE Transactions on Software Engineering, vol. 50, no. 1, hlm. 200–221, 2024, doi: 10.1109/TSE.2023.3325678.
K. B. Olesen dan J. Y. Lee, “Automated Software Engineering in the Cloud: CI/CD Fundamentals,” Applied Sciences, vol. 13, no. 4, hlm. 2450, 2023, doi: 10.3390/app13042450.
T. Dingsøyr dan N. B. Moe, “Accelerating Release Cycles with Advanced CI/CD Pipelines,” IEEE Softw., vol. 39, no. 2, hlm. 34–41, 2022, doi: 10.1109/MS.2021.3130112.
P. Anderson dan others, “Performance Evaluation of CI Tools in Agile Environments,” Empir. Softw. Eng., vol. 28, no. 5, hlm. 112, 2023, doi: 10.1007/s10664-023-10355-w.
S. Gupta dan D. Sharma, “Scaling Continuous Integration in Enterprise Microservices Architecture,” IEEE Access, vol. 12, hlm. 11020–11035, 2024, doi: 10.1109/ACCESS.2024.3351221.
A. Zaidman, “Characterizing Build Failures in Continuous Integration Systems,” Journal of Systems and Software, vol. 185, hlm. 111162, 2022, doi: 10.1016/j.jss.2021.111162.
W. Lam dan others, “Understanding and Handling Flaky Tests in CI/CD Environments,” IEEE Transactions on Software Engineering, vol. 49, no. 4, hlm. 1502–1520, 2023, doi: 10.1109/TSE.2022.3181122.
M. Beller dan others, “Bottlenecks in Continuous Deployment Pipelines: An Empirical Study,” Software Quality Journal, vol. 30, no. 1, hlm. 91–118, 2022, doi: 10.1007/s11219-021-09555-5.
H. Jin dan K. Liu, “An Empirical Study of CI Build Failures and Resolution Strategies,” Electronics (Basel)., vol. 12, no. 6, hlm. 1422, 2023, doi: 10.3390/electronics12061422.
R. Torkar dan others, “The Cost of Broken Builds in Agile Teams,” IEEE Trans. Eng. Manag., vol. 69, no. 5, hlm. 1820–1834, 2022, doi: 10.1109/TEM.2020.3015560.
E. Murphy-Hill dan others, “Impact of Build Breakages on Developer Productivity,” Inf. Softw. Technol., vol. 165, hlm. 107338, 2024, doi: 10.1016/j.infsof.2023.107338.
C. Bezemer, “Measuring the Efficiency of CI/CD Operations in Large-Scale Repositories,” IEEE Access, vol. 11, hlm. 55600–55615, 2023, doi: 10.1109/ACCESS.2023.3281145.
F. Fagerholm dan others, “Delays and Disruptions in Automated Delivery Pipelines,” Empir. Softw. Eng., vol. 27, no. 4, hlm. 95, 2022, doi: 10.1007/s10664-022-10156-1.
J. Penix, “Limitations of Static Rule-Based CI Monitoring,” IEEE Softw., vol. 40, no. 3, hlm. 22–28, 2023, doi: 10.1109/MS.2023.3245110.
Y. Wang dan Z. Zhang, “Manual vs Automated Monitoring in DevOps,” Information, vol. 13, no. 8, hlm. 385, 2022, doi: 10.3390/info13080385.
S. H. Setiyani dan Y. Rahma, “Penerapan Dekomposisi Matriks untuk Reduksi Kompleksitas Komputasi pada Algoritma Machine Learning,” Apr 2026, doi: https://doi.org/10.37859/jf.v16i1.11166.
A. Navidkya dan M. Yusuf, “Analisis Sentimen Isu Artificial Intelligence di Twitter dengan SVM dan Random Forest,” Apr 2026, doi: https://doi.org/10.37859/jf.v16i1.10037.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Rizqia Fauziah Rachma, Windu Gata, Farizal Ginanjar, Muhammad Arief Nadhofa
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright Notice
An author who publishes in the Jurnal FASILKOM (teknologi inFormASi dan ILmu KOMputer) agrees to the following terms:
- Author retains the copyright and grants the journal the right of first publication of the work simultaneously licensed under the Creative Commons Attribution-ShareAlike 4.0 License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal
- Author is able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book) with the acknowledgement of its initial publication in this journal.
- Author is permitted and encouraged to post his/her work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of the published work (See The Effect of Open Access).
Read more about the Creative Commons Attribution-ShareAlike 4.0 Licence here: https://creativecommons.org/licenses/by-sa/4.0/.
