Testing Risk Management Decision Making Competency of Project Managers in a Crisis

Summary

Strang and Vajjhala (2022) tested 24 experienced US project managers in a repeated-measures controlled quasi-experiment on how cognitive bias and individual factors affect the decision to cancel a failing project in a crisis. A biasing incentive significantly lowered decision quality, and certification and risk competency were the only individual factors significantly related to good decisions (rho = 0.549 to 0.712). Competent, certified project managers made the best decisions under bias, whereas a certified but incompetent project manager failed to cancel. The authors caution against generalizing because the study was exploratory.

Practical implication. For recruiters and employers of project managers, certification alone does not protect against bias in risk decisions, so competency testing, risk management training and ethics training are recommended.

The Problem

Failing projects are often not cancelled in a crisis, and it is unclear how much cognitive bias and individual factors such as certification, competency and experience drive project managers’ risk decisions.

The Methodology

Repeated-measures controlled quasi-experiment with 24 experienced U.S. project managers (competency test, basic decision, biased decision) analysed with Spearman correlation, MANOVA, repeated-measures ANOVA, Friedman test and post-hoc t-tests.

The Core Finding

In a controlled experiment with 24 experienced US project managers, a biasing incentive significantly lowered decision quality, and certification and risk competency were the only individual factors significantly related to good decisions (rho = 0.549 to 0.712). Competent, certified project managers made the best decisions under bias, whereas a certified but incompetent project manager failed to cancel.

The Citation

Strang, K. D., & Vajjhala, N. R. (2022). Testing Risk Management Decision Making Competency of Project Managers in a Crisis. The Journal of Modern Project Management, 10(1), 52–71. https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904 Copy APA

The summary, key findings, methods, implications and relevance notes on this page are this website’s own description of the paper, written from the published abstract and text. The official abstract and citation details are given further down.

The study in detail

What question does this paper answer?

How much do cognitive bias and other individual factors (age, gender, education, experience, certification and competency) affect a project manager's risk management decision to cancel a failing project during a crisis?

What did the study find, in detail?

In a repeated-measures controlled experiment with 24 experienced U.S.-based project managers, bias (a 400% bonus offered to keep a failing project going) significantly lowered the quality of decisions, while certification and competency were the only individual factors significantly related to good decisions (e.g., certification rho = 0.549 and 0.712; competency rho = 0.680 and 0.681). Competent and certified PMs made the best decisions in both the basic (M = 4.8) and biased (M = 4.2) conditions, whereas a certified but incompetent PM failed to cancel under bias (M = 1.7).

Why does it matter?

The authors argue that bias such as money and ego can override ethical risk management judgment even among certified project managers, which could be a problem for recruiters who rank certification highly without seeking evidence of competency. They suggest certified PMs receive additional risk management training, that certification courses include experiential exercises on quantifying and mitigating risks, and that employers offer short ethics or organizational refresher courses.

This study is part of the evidence overview What predicts IT project failure and success? Evidence on project managers, bias and team transparency from machine learning and experiments

Methods and data

Definitions and key terms

Risk management decision making

The emotional or thinking processes a project manager uses to solve complex problems in a project beyond business-as-usual activities, such as deciding whether to escalate or cancel a failing project after a severe risk event. — as used in Strang & Vajjhala (2022)

Prospect theory

A model of decision making under risk in which choices are made between value prospects or gambles, shaped by loss aversion, reference points and decision weights. — as used in Strang & Vajjhala (2022)

Repeated measures design

An experimental design in which every participant receives all treatments, so individual differences remain the only varying factors. — as used in Strang & Vajjhala (2022)

All key terms across the publications →

Abstract

The objective of the current study was to use a rigorous controlled experiment simulating a project failure to measure how cognitive bias and competency impact a PM’s risk management decision making in a crisis while controlling for other project and firm level variables including lying or faking responses. The MANOVA, repeated measures ANOVA controlled experiment and post-hoc analysis techniques were rigorous because the study took place in approximately the same point in time and each participant received all treatments. The 24 respondents in these repeated measures experiment outperforms most psychology factorial research design, which would require a 4 x 24 = 96 sample size to accommodate 3 treatments and a control group. We found bias significantly impacted PM risk management decision making in a crisis, but certification and competency resulted in the best decisions. Generalizations are cautioned due to the exploratory nature of this study. However, the literature review and methods were articulated well enough to encourage replications and extensions by other researchers.

Abstract as published in The Journal of Modern Project Management.

Keywords: Project management; risk management; project manager; decision making; crisis; project failure; repeated measures ANOVA; experiment; MANOVA

Limitations and open questions

• Generalizations must be cautioned because this was a controlled experiment with only 24 participants based in the U.S.
• Certification was self-reported and competency was judged solely by the score on one open-book PERT risk calculation test question.
• All organization-level factors were controlled, so the experimental context may not be equivalent to real-world conditions, reducing generalizability.
• The bias condition combined money and implied ego; other bias constructs were not tested.
• Competency scores were transformed into a binary pass/fail field (60% pass) to allow ANOVA and MANOVA; MANCOVA with continuous covariates is recommended.
• The study was a quasi-experiment conducted asynchronously online, participants self-selected, and lingering sentiments from the first treatment may have affected the second.

Open questions

• Do other forms of cognitive bias beyond money and implied ego affect project termination decisions, and do results hold with larger, non-U.S. samples and MANCOVA designs?

Open questions from the evidence overview Project management and project success.

Practical implications

• Recruiters who rank certification highly should also seek evidence of competency, because the authors argue that bias such as money and ego can override ethical risk management judgment even among certified project managers.
• Certified project managers should receive additional risk management training, and certification courses should include experiential exercises on quantifying and mitigating risks, the authors suggest.
• Employers should offer short ethics or organizational refresher courses, as the authors suggest.
• Researchers should cite the study as experimental evidence rather than a population estimate, because the sample size is small.

Drawn from Strang & Vajjhala (2022), The Journal of Modern Project Management, 10(1), pp. 52–71.

When this research may be relevant

This article may be relevant to researchers studying cognitive bias and prospect theory in project decisions, risk management competency versus certification, decisions to terminate failing projects, crisis decision-making, and experimental designs in project management research. Sample size is small, so it is best cited as experimental evidence rather than a population estimate.

Research topics addressed: project management; risk management; project failure; decision making; cognitive bias; project manager competency; PM certification; controlled experiment; repeated measures ANOVA; MANOVA; prospect theory; COVID-19; PERT estimation; United States

How to cite

APA 7

Strang, K. D., & Vajjhala, N. R. (2022). Testing Risk Management Decision Making Competency of Project Managers in a Crisis. The Journal of Modern Project Management, 10(1), 52–71. https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904Copy APA citation

MLA 9

Strang, Kenneth David, and Narasimha Rao Vajjhala. “Testing Risk Management Decision Making Competency of Project Managers in a Crisis.” The Journal of Modern Project Management, vol. 10, no. 1, 2022, pp. 52–71. https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904.Copy MLA citation

IEEE

K. D. Strang and N. R. Vajjhala, “Testing Risk Management Decision Making Competency of Project Managers in a Crisis,” The Journal of Modern Project Management, vol. 10, no. 1, pp. 52–71, 2022. [Online]. Available: https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904Copy IEEE citation

@article{strang2022projectmanager,
  title = {Testing Risk Management Decision Making Competency of Project Managers in a Crisis},
  author = {Strang, Kenneth David and Vajjhala, Narasimha Rao},
  journal = {The Journal of Modern Project Management},
  volume = {10},
  number = {1},
  pages = {52--71},
  year = {2022},
  url = {https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904}
}

TY  - JOUR
TI  - Testing Risk Management Decision Making Competency of Project Managers in a Crisis
AU  - Strang, Kenneth David
AU  - Vajjhala, Narasimha Rao
JO  - The Journal of Modern Project Management
PY  - 2022
DA  - 2022
VL  - 10
IS  - 1
SP  - 52
EP  - 71
UR  - https://www.narasimharao.net/research/project-manager-risk-decision-bias-crisis-experiment/
LA  - en
ER  - 

Links and files

• Publisher’s page: https://journalmodernpm.com/manuscript/index.php/jmpm/article/view/JMPM02904
• Full text (PDF): https://journalmodernpm.com/manuscript/index.php/jmpm/article/download/JMPM02904/451
• This page as text: https://www.narasimharao.net/research/project-manager-risk-decision-bias-crisis-experiment/paper.md
• Permanent address of this page: https://www.narasimharao.net/research/project-manager-risk-decision-bias-crisis-experiment/