Journal article · 2022
How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project
International Journal of Information Technology Project Management, 13(1), pp. 1–17 · Published
Summary
Strang and Vajjhala (2022) tested 16 experienced US IT project managers in an online repeated-measures controlled experiment to measure how hidden value prospect bias affects the decision not to terminate a simulated failing project. A 400% bonus incentive significantly lowered the mean likelihood of cancelling the project from 3.438 to 2.438 on a 1–5 scale (p = .005, Cohen’s d = 0.826). Risk competency, not certification, was the significant protective factor (η² = 0.218), though the findings should be generalized with caution because the experiment had only 16 participants.
Practical implication. Sponsors hiring project managers should require both verified certification and tested competency, because risk competency, not certification, was the significant protective factor in an experiment with 16 IT project managers.
- The Problem
- Competent project managers sometimes keep failing projects alive after a risk event, and the role of hidden value-prospect bias in that decision had not been measured experimentally.
- The Methodology
- Online repeated-measures controlled experiment with 16 experienced U.S. IT project managers in a simulated military software project, using a PERT competency test, basic and incentive-biased decision conditions, paired t-test and repeated-measures ANOVA.
- The Core Finding
- A 400% bonus incentive significantly lowered 16 experienced IT project managers’ likelihood of cancelling a failing project (from 3.438 to 2.438 on a 1–5 scale; p = .005, Cohen’s d = 0.826). Risk competency, not certification, was the significant protective factor (η² = 0.218).
- The Citation
- Strang, K. D., & Vajjhala, N. R. (2022). How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project. International Journal of Information Technology Project Management, 13(1), 1–17. https://doi.org/10.4018/IJITPM.304059
The summary, key findings, methods, implications and relevance notes on this page are this website’s own description of the paper, written from the published abstract and text. The official abstract and citation details are given further down.
Key findings
- Finding 1. The experiment tested 16 IT project managers (mean age 40.6, mean experience 9.8 years, 88% male, 44% certified), all based in the USA and working for small-to-medium organizations, using a simulated failing military IT software project.
- Finding 2. Fewer than half (43%) of the project managers passed the PERT-based risk estimation competency test; only 22% of non-certified project managers were competent compared with 71% of certified project managers.
- Finding 3. Offering a 400% bonus not to cancel reduced the mean likelihood of cancelling the failing project from 3.438 (SD = 1.672) to 2.438 (SD = 1.209) on a 1–5 scale, a significant difference (t = 3.303, df = 15, p = .005, Cohen’s d = 0.826).
- Finding 4. In the repeated measures ANOVA, project manager risk competency was a significant between-subjects factor (F = 7.861, p = .016, η² = 0.218), a 22% effect size, whereas certification was not significant (F = 1.956, p = .187).
- Finding 5. Competency correlated strongly with the decision to cancel a failing project in both the basic condition (Spearman rho = 0.824, p < .001) and the biased condition (rho = 0.631, p < .01), while age, gender and experience were not related to either decision.
- Finding 6. Incompetent but certified project managers cancelled in the basic scenario (M = 4.5) but not under the value prospect incentive (M = 1.5), whereas competent and certified project managers scored 4.5 and 3.6 respectively.
- Finding 7. Incompetent and uncertified project managers made the worst decisions, continuing the failing project in both the basic (M = 2) and biased (M = 1.7) conditions.
Source: Strang & Vajjhala (2022), International Journal of Information Technology Project Management, 13(1), pp. 1–17. DOI: 10.4018/IJITPM.304059
The study in detail
What question does this paper answer?
Why do some competent project managers not terminate a failing project after a risk event, and how much does hidden cognitive (value prospect) bias affect the decision not to end a troubled project?
What did the study find, in detail?
In a repeated measures experiment with 16 experienced US IT project managers, the mean likelihood of cancelling a failing project fell from 3.438 (SD = 1.672) in the basic condition to 2.438 (SD = 1.209) when a 400% bonus incentive was offered (t = 3.303, p = .005, Cohen’s d = 0.826). Risk competency was the significant factor (F = 7.861, p = .016, η² = 0.218, a 22% effect size), while certification was not (F = 1.956, p = .187).
Why does it matter?
Value prospect incentives (money and ego) can push even certified project managers to continue a doomed project against policy, which may explain some of the roughly 50% global project failure rate. The authors recommend that sponsors require both verified certification and tested competency when hiring project managers, and give existing project managers periodic refresher training in ethical decision making and risk estimation.
This study is part of the evidence overview What predicts IT project failure and success? Evidence on project managers, bias and team transparency from machine learning and experiments
Methods and data
| Research question | Why do some competent project managers not terminate a failing project after a risk event, and how much does cognitive bias affect that decision? |
|---|---|
| Design | Online repeated measures controlled experiment: a PERT risk-estimation competency test, then a basic (no incentive) and a biased (400% bonus) decision condition in a simulated military IT software project hit by COVID-19 |
| Sample | 16 project managers with at least 5 years’ experience, randomly drawn from about 23,000 professional-level PMs in a LinkedIn group; all based in the USA |
| Methods | Shapiro-Wilk tests, Spearman correlations, paired t-test and repeated measures ANOVA |
| Main result | Competency significantly predicted better decisions (F = 7.861, p = .016, η² = 0.218); certification did not (p = .187); the bias incentive lowered mean likelihood to cancel from 3.438 to 2.438 |
| Implication | Hire project managers on verified certification plus tested competency, and provide refresher training on ethical, policy-following risk decisions |
| Citation | Strang & Vajjhala (2022) · DOI 10.4018/IJITPM.304059 |
Definitions and key terms
- Prospect theory
- A model of risk decision making in which people choose between value prospects based on loss aversion, reference points and decision weights, tending to be risk-averse for losses and risk-seeking for gains (Kahneman and Tversky, 1979). — as used in Strang & Vajjhala (2022)
- Value prospect bias
- In this study, the influence of a personal economic and emotional gain (a 400% bonus) offered to a project manager for not cancelling a failing project. — as used in Strang & Vajjhala (2022)
- PERT estimate
- A three-point estimate combining optimistic, most likely and pessimistic durations, used here to test project manager risk-estimation competency (correct answer: 32 days). — as used in Strang & Vajjhala (2022)
Abstract
Almost half of projects have failed globally during the last 50 years, yet most studies in the literature review were inclusive. The research design was a robust repeated measure controlled experiment where the 16 participants received all treatments, which may be contrasted to a similar 4x4 factorial experiment with a control group (common in psychology or healthcare) resulting in a group size of only four. All but the individual project manager (PM) factors were controlled, while primary demographic and behavior data were collected. PMs were tested for competence using a risk management scenario and given two manipulated conditions (a basic and a biased treatment). Since the organizational and project level factors were controlled, some individual level factors impacted the decision. PMs with higher competence made better decisions, with a 22% effect size, when all other factors in the model were accounted for. Competent non-certified PMs made better decisions as compared to certified incompetent PMs.
Abstract as published in International Journal of Information Technology Project Management.
Keywords: Crisis; Decision Making Bias; Experiment; Project Failure; Project Management; Project Manager; Risk Management
Limitations and open questions
- Generalizations must be cautioned because the controlled experiment had only 16 participants.
- Certification was self-reported and not verified, and the specific type of certification was not contrasted.
- Competency was judged solely by the score on one open-book PERT risk calculation question, and was dichotomized (pass/fail) to allow repeated measures ANOVA.
- All organization-level factors were controlled and defined by the authors, so the project and organizational conditions may not match the real world.
- Generalizations are limited to IT project managers in the USA; participants were given only 15 minutes per decision.
Open questions
- Do other forms of cognitive bias beyond money and implied ego affect project termination decisions, and do results hold with larger, non-U.S. samples and MANCOVA designs?
- Do specific types of PM certification differ in their relationship with project outcomes, given that certification was self-reported and not contrasted by type?
Open questions from the evidence overview Project management and project success.
Practical implications
- When hiring project managers, sponsors should verify certification and also test competency: competent non-certified project managers made better decisions than certified incompetent ones.
- Sponsors should give existing project managers periodic refresher training in ethical decision making and risk estimation, as the authors recommend.
- For sponsors, value prospect incentives (money and ego) can push even certified project managers to continue a doomed project against policy.
Drawn from Strang & Vajjhala (2022), International Journal of Information Technology Project Management, 13(1), pp. 1–17.
When this research may be relevant
This article may be relevant to literature on escalation of commitment and the decision not to terminate failing projects, cognitive bias in project managers, competency-based versus certification-based hiring, and experimental methods in IT project management. It complements the companion crisis-competency experiment listed under related research.
Research topics addressed: project failure; project management; cognitive bias; decision making; risk management; project termination; prospect theory; project manager competency; PM certification; repeated measures experiment; ANOVA; IT projects; COVID-19; USA
How to cite
- APA 7
- Strang, K. D., & Vajjhala, N. R. (2022). How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project. International Journal of Information Technology Project Management, 13(1), 1–17. https://doi.org/10.4018/IJITPM.304059
- MLA 9
- Strang, Kenneth David, and Narasimha Rao Vajjhala. “How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project.” International Journal of Information Technology Project Management, vol. 13, no. 1, 2022, pp. 1–17. https://doi.org/10.4018/IJITPM.304059.
- IEEE
- K. D. Strang and N. R. Vajjhala, “How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project,” International Journal of Information Technology Project Management, vol. 13, no. 1, pp. 1–17, Jul. 2022, doi: 10.4018/IJITPM.304059.
BibTeX
@article{strang2022projectmanager2,
title = {How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project},
author = {Strang, Kenneth David and Vajjhala, Narasimha Rao},
journal = {International Journal of Information Technology Project Management},
volume = {13},
number = {1},
pages = {1--17},
year = {2022},
publisher = {IGI Global},
doi = {10.4018/IJITPM.304059},
url = {https://doi.org/10.4018/IJITPM.304059}
}RIS
TY - JOUR TI - How Bias Impacted the Project Manager Decision to Not Terminate a Failing Project AU - Strang, Kenneth David AU - Vajjhala, Narasimha Rao JO - International Journal of Information Technology Project Management PY - 2022 DA - 2022/07/13 VL - 13 IS - 1 SP - 1 EP - 17 PB - IGI Global DO - 10.4018/IJITPM.304059 UR - https://www.narasimharao.net/research/project-manager-bias-failing-project-termination-experiment/ LA - en ER -
Links and files
Related work
Related work: the literature this study cites
The study cites large empirical studies of defense procurement projects: Eckerd and Snider analysed 1,073 Department of Defense projects and found no significant correlation between contractor factors such as experience, PM certification, age or gender and cost variance or breach[1], and Borbath and colleagues examined 14,836 contractor projects and none of their hypotheses were supported[2]. Strang’s logistic regression of 2,692 defense IT projects found several significant individual and organizational factors, yet they accounted for only 12% of the variance in outcomes[3]. The experiment is grounded in prospect theory, in which decisions are framed as choices between value prospects shaped by loss aversion and reference points[4], and in findings that heuristics, herding and prospect biases, with overconfidence as the dominant heuristic, can produce irrational investment choices[5]. Klein and colleagues’ argument that crisis decision-making relies on tacit competencies that are hard to teach through training or certification is used to motivate testing competency directly rather than relying on credentials[6].
- [1] Eckerd, A., & Snider, K. (2017). Does the program manager matter? New public management and defense acquisition. American Review of Public Administration, 47(1), 36–57. https://doi.org/10.1177/0275074015596376
- [2] Borbath, M., Blessner, P., & Olson, B. (2019). An empirical approach to the evaluation of defence contractor performance. International Journal of Logistics Research and Applications, 22(2), 138–153. https://doi.org/10.1080/13675567.2018.1497148
- [3] Strang, K. D. (2021). Which organizational and individual factors predict success versus failure in procurement projects. International Journal of Information Technology Project Management, 12(3), 19–39. https://doi.org/10.4018/IJITPM.2021070102
- [4] Kahneman, D., & Tversky, A. (1979). Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), 263-292. https://doi.org/10.2307/1914185
- [5] Goyal, P., Gupta, P., & Yadav, V. (2021). Antecedents to heuristics: Decoding the role of herding and prospect theory for Indian millennial investors. Review of Behavioral Finance, 13(6). Advance online publication. https://doi.org/10.1108/RBF-04-2021-0073
- [6] Klein, D. E., Woods, D. D., Klein, G. A., & Perry, S. J. (2016). Can we trust best practices? Six cognitive challenges of evidence-based approaches. Journal of Cognitive Engineering and Decision Making, 10(3), 244–254. https://doi.org/10.1177/1555343416637520
From the evidence overview What predicts IT project failure and success? Evidence on project managers, bias and team transparency from machine learning and experiments, which cites only works that appear in the studies’ own reference lists. This is a selection, not the full reference list of Strang & Vajjhala (2022).
Related research
Related publications on project management and project success
In a controlled experiment with 24 experienced US project managers, a biasing incentive significantly lowered decision quality, and certification and risk competency were the only individual factors significantly related to good decisions (rho = 0.549 to 0.712).
Across 439 successful US government IT projects with 1,230 features, linear regression outperformed random forest and SVM in predicting project manager commitment tenure.
Team members’ willingness to disclose their past performance evaluations during procurement significantly improved project success after controlling for budget, end-user community size and certification, roughly doubling the explained variance in project success from 9.6% to 18.8%.
Supervised machine learning on 439 successful US government IT projects showed linear regression best predicted project manager commitment tenure (r² = 0.238), with line of business, experience, network collaboration tools and cross-industry team membership as the strongest features — supporting an association between higher commitment and project success.
The volume compiles evidence on risk and contingency management across nations and industries — from petroleum and automotive risk models to cybersecurity, healthcare informatics and educational inequality — offering business leaders lessons for future crises.