What predicts IT project failure and success? Evidence on project managers, bias and team transparency from machine learning and experiments

Context: the persistent IT project failure rate and small effect sizes

The starting point of this research programme is that roughly half of IT-related projects fail and that this rate has changed little over decades (Strang & Vajjhala, 2023, IJITPM) (Strang & Vajjhala, 2022, IJITPM). The papers cite the Standish Group's 2009 figure that only 32% of U.S. government projects were successful^[1], and large empirical studies of defense and public procurement projects: Eckerd and Snider analysed 1,073 Department of Defense projects and found no significant correlation between contractor factors such as experience, PM certification, age or gender and cost variance or breach^[2]; Borbath and colleagues examined 14,836 contractor projects and none of their hypotheses were supported^[3]; and Ghossein and colleagues reported a 50% failure rate across 59,816 European Union public procurement projects, with small effect sizes^[4]. Strang's logistic regression of 2,692 defense IT projects found several significant individual and organizational factors, yet they accounted for only 12% of the variance in outcomes^[5].

Vajjhala and colleagues argue that two methodological habits contribute to these weak results: reliance on surveys that ask for opinions about project performance long after the fact, and single case studies that are hard to generalise (Strang & Vajjhala, 2023, IJITPM) (Strang & Vajjhala, 2022, IJITPM). Their response is twofold: mining retrospective project records with machine learning, and running controlled experiments that isolate the project manager's risk decision.

What the machine learning studies of project failure and PM commitment found

In the project failure prediction study, U.S. Government Accountability Office data were cleaned and a random forest (94 trees, 4 predictors per split) was trained on a binary breach/failure outcome, reaching precision 0.799, recall 0.810 and F1 0.798 (Strang & Vajjhala, 2023, IJITPM). Seven features passed the authors' node-purity cutoff of 0.02, led by PM experience (0.063) and contract versus in-house PM (0.058). In the post-hoc logistic regression, more PM experience was associated with avoiding failure (beta = -1.441), higher budgets with breach (beta = 0.744), and in-house rather than outsourced PMs with breached projects, while PM salary (p = 0.278) and PM certification (p = 0.591) were not significant (Strang & Vajjhala, 2023, IJITPM). The authors compare their random forest metrics with Han, Lung and Ajila's software defect prediction study, which reported a random forest ROC of 0.844^[9], and they draw on Pospieszny and colleagues' argument that machine learning on historical data can reduce technical, psychological and political bias in project estimates^[10].

A second line of work used 1,230 features from over 500 U.S. (mainly military) IT projects in 2015-2023, cleaned to 439 successful projects, to predict project manager organizational commitment, measured as tenure with the same employer (mean 11.08 years) (Strang & Vajjhala, 2025, IJPOM) (Vajjhala & Strang, 2024, WorldCIST 2024). Clustering was weak (silhouette scores 0.18 to 0.2), so the authors moved to regression: linear regression explained 23.8% of the variance in commitment tenure (MAE = 5.3051), compared with 10.8% for random forest and 4.7% for SVM (Strang & Vajjhala, 2025, IJPOM). Line of business (relative relief 0.36), experience (0.29), network collaboration type (0.28), cross-industry subject matter (0.27) and contract status (0.22) were the strongest features (Strang & Vajjhala, 2025, IJPOM). The conference paper and the journal article report the same analysis, so they are one piece of evidence rather than two (Vajjhala & Strang, 2024, WorldCIST 2024) (Strang & Vajjhala, 2025, IJPOM). The work frames commitment through Meyer and Allen's affective, continuance and normative components^[12] and positions itself after earlier predictive uses of machine learning in project control, such as Wauters and Vanhoucke's support vector machine regression for forecasting^[11].

Experimental evidence on project manager cognitive bias and project termination

Two repeated-measures experiments simulated a failing military software project hit by a COVID-19 risk event, first asking PMs whether to cancel with no incentive and then with a 400% bonus offered for not cancelling. With 16 experienced U.S. IT project managers, the bias condition significantly lowered the likelihood of cancelling (t = 3.303, p = .005, Cohen's d = 0.826); tested PERT-based risk competency was a significant factor (F = 7.861, p = .016, eta2 = 0.218) while certification was not (p = .187) (Strang & Vajjhala, 2022, IJITPM). Certified but incompetent PMs cancelled in the basic scenario (M = 4.5) but not under the incentive (M = 1.5) (Strang & Vajjhala, 2022, IJITPM). With 24 participants, both certification (rho = 0.549 basic, 0.712 biased) and competency (rho = 0.680 and 0.681) were related to good decisions, and competent and certified PMs scored best (M = 4.8 basic, M = 4.2 biased) (Strang & Vajjhala, 2022, JMPM).

The experiments are grounded in prospect theory, in which decisions are framed as choices between value prospects shaped by loss aversion and reference points^[13], and in findings that heuristics, herding and prospect biases, with overconfidence as the dominant heuristic, can produce irrational investment choices^[14]. Klein and colleagues' argument that crisis decision-making relies on tacit competencies that are hard to teach through training or certification is used to motivate testing competency directly rather than relying on credentials^[15] (Strang & Vajjhala, 2022, IJITPM).

Where the evidence agrees, conflicts, and is limited

Certification is the least consistent predictor. It was not significant in the big-data failure model (Strang & Vajjhala, 2023, IJITPM) or in the 16-PM experiment (Strang & Vajjhala, 2022, IJITPM), but it was significant in the 24-PM experiment (Strang & Vajjhala, 2022, JMPM) and had a small positive coefficient in the Fintech study (B = 6.534, part correlation 0.081) (Strang & Vajjhala, 2025, Information). The papers note the same split in prior work: Catanio and colleagues found no difference between certified and uncertified PMs in scope, time and cost management^[6], and Nazeer and Marnewick are cited among studies finding no certification effect on project success^[7]. Experience also shows a context split: it was the strongest failure predictor in retrospective data (Strang & Vajjhala, 2023, IJITPM), but was not related to the cancel decision in either experiment (Strang & Vajjhala, 2022, IJITPM) (Strang & Vajjhala, 2022, JMPM), which may reflect the different outcomes measured. Budget pointed the same way in both datasets, with higher budgets linked to breach (Strang & Vajjhala, 2023, IJITPM) and lower Fintech project success (B = -0.224) (Strang & Vajjhala, 2025, Information), consistent with reviews linking budget and complexity to lower success^[17].

In the Fintech study, 518 team members (41.5% response rate) answered whether they would share their previous performance evaluation; willingness correlated with outcome (r = 0.364) and was the strongest predictor in the final model (part correlation 0.341), while leadership, culture, stakeholder management, change management and experience were not significant (Strang & Vajjhala, 2025, Information). The authors caution that willingness was measured after projects ended, so the association is correlational, that the data came from one U.S. company, and that about 40% of initiated projects were cancelled and absent from the records (Strang & Vajjhala, 2025, Information). Other limitations include the 94% male PM population in the failure data (Strang & Vajjhala, 2023, IJITPM), uncertainty about whether all projects were military (Strang & Vajjhala, 2025, IJPOM), single-question competency tests and self-reported certification (Strang & Vajjhala, 2022, IJITPM) (Strang & Vajjhala, 2022, JMPM), and a 24-person sample that was expanded from an initial shortlist of 16, so the two experiments are not fully independent (Strang & Vajjhala, 2022, JMPM). For comparison, a LinkedIn-sampled survey of 625 PMs by Brandon and colleagues found experience factors predicted percent success with r2 = 0.025^[16], while Saadé and colleagues' UN agency survey identified engagement, education and experience factors^[8].

Evidence table

Questions researchers ask

Does project management certification reduce IT project failure?

The evidence here is mixed. Certification was not significant in a logistic regression of big-data IT project breaches (p = 0.591) (Strang & Vajjhala, 2023, IJITPM) or in a 16-PM experiment (p = .187) (Strang & Vajjhala, 2022, IJITPM), but it was related to good cancel decisions in a 24-PM experiment (Strang & Vajjhala, 2022, JMPM). Prior studies cited by the authors also disagree^[6][7].

Can machine learning predict IT project failure?

In one U.S. government dataset, a random forest reached precision 0.799, recall 0.810 and a ROC area of 0.849, and a follow-up logistic regression on seven features had a McFadden r2 of 0.267 (Strang & Vajjhala, 2023, IJITPM). The authors describe the list of indicators as preliminary and call for replication with other techniques and countries (Strang & Vajjhala, 2023, IJITPM).

Why do project managers fail to cancel failing projects?

In two repeated-measures experiments, a 400% bonus incentive lowered the quality of cancel decisions, and managers who passed a PERT-based risk competency test made better decisions under both conditions (Strang & Vajjhala, 2022, IJITPM) (Strang & Vajjhala, 2022, JMPM). The authors interpret this through prospect theory and value prospect bias^[13], but note the small U.S. samples limit generalization (Strang & Vajjhala, 2022, IJITPM).

Does team transparency affect project success?

In a Fintech company, team members willing to share past performance evaluations were associated with higher project success scores (r = 0.364), and adding willingness raised explained variance from 9.6% to 18.8% (Strang & Vajjhala, 2025, Information). Because willingness was measured after projects ended in a single firm, the authors treat the result as correlational (Strang & Vajjhala, 2025, Information).

Open questions

• Do the failure indicators replicate outside the U.S. and with other machine learning techniques, using project performance records rather than surveys (Strang & Vajjhala, 2023, IJITPM)?
• How does project manager commitment tenure behave in failed projects, since only successful projects were examined (Strang & Vajjhala, 2025, IJPOM)?
• Would willingness to disclose past performance, measured before team selection in longitudinal or quasi-experimental designs, still predict success across industries and privacy regimes such as GDPR (Strang & Vajjhala, 2025, Information)?
• Do other forms of cognitive bias beyond money and implied ego affect project termination decisions, and do results hold with larger, non-U.S. samples and MANCOVA designs (Strang & Vajjhala, 2022, JMPM) (Strang & Vajjhala, 2022, IJITPM)?
• Do specific types of PM certification differ in their relationship with project outcomes, given that certification was self-reported and not contrasted by type (Strang & Vajjhala, 2022, IJITPM)?

References

Studies summarised

• Strang, Vajjhala (2023). Mining Project Failure Indicators From Big Data Using Machine Learning Mixed Methods. International Journal of Information Technology Project Management, 14(1), pp. 1–24. https://doi.org/10.4018/IJITPM.317221
• Strang, Vajjhala (2025). Exploring project manager commitment using machine learning on fuzzy big data. International Journal of Project Organisation and Management, 17(2), pp. 135–152. https://doi.org/10.1504/IJPOM.2025.146727
• Vajjhala, Strang (2024). An Exploratory Big Data Approach to Understanding Commitment in Projects. WorldCIST 2024 (World Conference on Information Systems and Technologies), Lecture Notes in Networks and Systems, vol. 989, pp. 66–75, Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-60227-6_6
• Strang, Vajjhala (2022). How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project. International Journal of Information Technology Project Management, 13(1), pp. 1–17. https://doi.org/10.4018/IJITPM.304059
• Strang, Vajjhala (2022). Testing Risk Management Decision Making Competency of Project Managers in a Crisis. The Journal of Modern Project Management, 10(1), pp. 52–71. https://doi.org/10.19255/JMPM02904
• Strang, Vajjhala (2025). Can Project Team Members’ Willingness to Disclose Past Performance During Procurement Improve Organizational Business Process Success?. Information, 16(11), Article 955. https://doi.org/10.3390/info16110955

Other literature cited

1. [1] Standish-Group. (2009). Chaos Summary 2009: The 10 Laws of CHAOS. University of Chicago. https://www.classes.cs.uchicago.edu/archive/2014/fall/51210-1/required.reading/Standish.Group.Chaos.2009.pdf
2. [2] Eckerd, A., & Snider, K. (2017). Does the program manager matter? New public management and defense acquisition. American Review of Public Administration, 47(1), 36–57. doi:10.1177/0275074015596376 https://doi.org/10.1177/0275074015596376
3. [3] Borbath, M., Blessner, P., & Olson, B. (2019). An empirical approach to the evaluation of defence contractor performance. International Journal of Logistics Research and Applications, 22(2), 138–153. doi:10.1080/13675567.2018.1497148 https://doi.org/10.1080/13675567.2018.1497148
4. [4] Ghossein, T., Islam, A. M., & Saliola, F. (2018). Public procurement and the private business sector: Evidence from firm-level data. In K. V. Thai, P. J. Salia, & F. A. Mwakibinga (Eds.), Global public procurement: Theories and practices (pp. 21–55). Springer. doi:10.1596/1813-9450-8575 https://doi.org/10.1596/1813-9450-8575
5. [5] Strang, K. D. (2021). Which organizational and individual factors predict success versus failure in procurement projects. International Journal of Information Technology Project Management, 12(3), 19–39. doi:10.4018/IJITPM.2021070102 https://doi.org/10.4018/IJITPM.2021070102
6. [6] Catanio, J. T., Armstrong, G., & Tucker, J. (2013). The effects of project management certification on the triple constraint. International Journal of Information Technology Project Management, 42(1), 31–41. doi:10.4018/ijitpm.2013100106 https://doi.org/10.4018/ijitpm.2013100106
7. [7] Nazeer, J., & Marnewick, C. (2018). Investing in project management certification: Do organisations get their moneys worth? Information Technology Management, 19(1), 51–74. doi:10.1007/s10799-017-0275-y https://doi.org/10.1007/s10799-017-0275-y
8. [8] Saadé, R. G., Dong, H., & Wan, H. (2015). Factors of project manager success. Interdisciplinary Journal of Information, Knowledge, and Management, 10(1), 63–80. doi:10.28945/2265 https://doi.org/10.28945/2265
9. [9] Han, W., Lung, C.-H., & Ajila, S. (2016). Using source code and process metrics for defect prediction - A case study of three algorithms and dimensionality reduction. Journal of Software, 11(9), 883–902. doi:10.17706/jsw.11.9.883-902 https://doi.org/10.17706/jsw.11.9.883-902
10. [10] Pospieszny, P., Czarnacka-Chrobot, B., & Kobylinski, A. (2018). An effective approach for software project effort and duration estimation with machine learning algorithms. Journal of Systems and Software, 137(2), 184–196. doi:10.1016/j.jss.2017.11.066 https://doi.org/10.1016/j.jss.2017.11.066
11. [11] Wauters, M. and Vanhoucke, M. (2014) ‘Support vector machine regression for project control forecasting’, Automation in Construction, Vol. 47, No. 1, pp.92–106.
12. [12] Meyer, J.P. and Allen, N.J. (1991) ‘A three-component conceptualization of organizational commitment’, Human Resource Management Review, Vol. 1, No. 1, pp.61–89.
13. [13] Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), 263-292. https://doi.org/10.2307/1914185
14. [14] Goyal, P., Gupta, P., & Yadav, V. (2021). Antecedents to heuristics: Decoding the role of herding and prospect theory for Indian millennial investors. Review of Behavioral Finance, 13(6). Advance online publication. doi:10.1108/RBF-04-2021-0073 https://doi.org/10.1108/RBF-04-2021-0073
15. [15] Klein, D. E., Woods, D. D., Klein, G. A., & Perry, S. J. (2016). Can we trust best practices? Six cognitive challenges of evidence-based approaches. Journal of Cognitive Engineering and Decision Making, 10(3), 244–254. doi:10.1177/1555343416637520 https://doi.org/10.1177/1555343416637520
16. [16] Brandon, T. P., Allen, B. M., & Menefee, S. J. (2022). Which Experience Factors Predict Overall Expertise and Percent Success? The Journal of Modern Project Management, 9(3), 20-31. https://journalmodernpm.com/index.php/jmpm/article/download/JMPM02802/8
17. [17] Iriarte, C.; Bayona, S. IT projects success factors: A literature review. Int. J. Inf. Syst. Proj. Manag. 2020, 8, 49–78.