An Exploratory Big Data Approach to Understanding Commitment in Projects

Summary

Vajjhala and Strang (2024) analyzed 439 successful US government IT projects with 1,230 features from a declassified repository, in an exploratory machine learning study of whether project performance indicators can be a reliable gauge of project manager (PM) organizational commitment. Linear regression best predicted PM commitment tenure (r² = 0.238), ahead of random forest (r² = 0.108) and SVM (r² = 0.047). Line of business, PM experience, network collaboration tools and cross-industry team membership were the strongest features, and the authors conclude that the findings support the proposition that higher PM commitment is associated with successful projects.

Practical implication. To inform talent retention, stakeholders in projects and programs can gauge project manager commitment from behavioral big data rather than self-reported surveys, on exploratory evidence from 439 successful US IT projects.

The Problem

Project manager commitment is usually gauged through biased self-reports, and it is unknown whether retrospective project big data can serve as an objective indicator.

The Methodology

Exploratory pragmatic machine learning study of a declassified U.S. government IT project repository (439 successful projects, 1,230 features) using radial analysis, multidimensional scaling and supervised regression (linear, random forest, SVM) with relief-based feature ranking.

The Core Finding

Supervised machine learning on 439 successful US government IT projects showed linear regression best predicted project manager commitment tenure (r² = 0.238), with line of business, experience, network collaboration tools and cross-industry team membership as the strongest features — supporting an association between higher commitment and project success.

The Citation

Vajjhala, N. R., & Strang, K. D. (2024). An Exploratory Big Data Approach to Understanding Commitment in Projects. In Á. Rocha (Ed.), WorldCIST 2024 (World Conference on Information Systems and Technologies) (pp. 66–75). Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-60227-6_6 Copy APA

The summary, key findings, methods, implications and relevance notes on this page are this website’s own description of the paper, written from the published abstract and text. The official abstract and citation details are given further down.

The study in detail

What question does this paper answer?

Can project performance indicators in retrospective big data serve as a reliable, bias-free gauge of project manager organizational commitment, and which attributes of successful projects predict PM commitment tenure?

What did the study find, in detail?

Analyzing 439 successful U.S. (primarily military) IT projects from 2015–2023 with supervised machine learning, the linear regression model best predicted PM commitment tenure (r² = 0.238, MAE = 5.31), ahead of random forest (r² = 0.108) and SVM (r² = 0.047). Line of business, PM experience, network collaboration tools and cross-industry team membership were the strongest features, supporting the proposition that higher PM commitment is associated with successful projects.

Why does it matter?

The study offers a more objective, big data-driven way to understand project manager commitment based on actual behavioral evidence rather than self-reported surveys, which suffer from bias and small effect sizes. Its insights are aimed at global stakeholders in projects and programs concerned with retaining experienced talent and reducing high project failure rates.

This study is part of the evidence overview What predicts IT project failure and success? Evidence on project managers, bias and team transparency from machine learning and experiments

Methods and data

Definitions and key terms

Organizational commitment

An employee's psychological attachment and loyalty to their organization; in this study, PM tenure was the critical measure of commitment. — as used in Vajjhala & Strang (2024)

Machine learning (ML)

A subset of artificial intelligence that lets computer systems perform tasks through algorithms and statistical models without explicit instructions, relying on pattern recognition and inference. — as used in Vajjhala & Strang (2024)

Coefficient of determination (r²)

The proportion of variance in the target variable explained by a model, used here as the effect size of each ML regression model. — as used in Vajjhala & Strang (2024)

All key terms across the publications →

Abstract

This study addresses the twin challenges of talent retention and high project failure rates (40–70%) by harnessing machine learning (ML) techniques to analyze retrospective big data. The study’s objective was to ascertain whether project performance indicators can be a reliable gauge of project manager (PM) organizational commitment. This approach sidesteps the inherent bias and small effect sizes associated with survey self-report responses. Our innovative methodology leverages secondary big data, transforming the values into structured features that predict PM organizational commitment. This study proposes a novel conceptual framework, focusing on actual behavioral evidence rather than traditional, self-reported attitudes to assess the fuzzy predictors of organizational commitment. Among the three developed ML models, one demonstrated a significant 24% effect size, uncovering key features correlating PM tenure and organizational commitment with success. The insights gained from this research have broad implications for global stakeholders in projects and programs, offering a more objective and big data-driven understanding of PM commitment.

Abstract as published in WorldCIST 2024 (World Conference on Information Systems and Technologies).

Keywords: Project Management; Organizational Commitment; Machine Learning; Big Data Analysis; Talent Retention; Project Failure Rates; Data-Driven Management; Predictive Modeling; Stakeholder Implications; Program Management; Performance Indicators

Limitations

• The data were retrospective secondary data, and it was ambiguous whether all IT projects involved military personnel or subcontractors.
• The study needs to be replicated using different samples and with triangulation of methods beyond machine learning.

Practical implications

• Stakeholders in projects and programs can use actual behavioral evidence in project big data as a more objective way to understand project manager commitment than self-reported surveys, which suffer from bias and small effect sizes.
• For stakeholders concerned with retaining experienced talent, the strongest features predicting project manager commitment tenure in 439 successful US government IT projects were line of business, project manager experience, network collaboration tools and cross-industry team membership.
• Stakeholders in projects and programs should treat the findings as exploratory, because the study needs to be replicated using different samples and with triangulation of methods beyond machine learning.

Drawn from Vajjhala & Strang (2024), WorldCIST 2024 (World Conference on Information Systems and Technologies), Lecture Notes in Networks and Systems, vol. 989, pp. 66–75, Springer Nature Switzerland AG.

When this research may be relevant

This article may be relevant to literature on project manager organisational commitment and retention measured from behavioural big data, machine learning in project management research, and predictors of commitment in successful government IT projects. It is the conference-stage version of the journal article on the same dataset listed under related research.

Research topics addressed: machine learning; big data; project management; organizational commitment; project manager tenure; talent retention; project success; IT projects; U.S. military projects; linear regression; random forest; support vector machine; multi-dimensional scaling; secondary data

How to cite

APA 7

Vajjhala, N. R., & Strang, K. D. (2024). An Exploratory Big Data Approach to Understanding Commitment in Projects. In Á. Rocha (Ed.), WorldCIST 2024 (World Conference on Information Systems and Technologies) (pp. 66–75). Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-60227-6_6Copy APA citation

MLA 9

Vajjhala, Narasimha Rao, and Kenneth David Strang. “An Exploratory Big Data Approach to Understanding Commitment in Projects.” WorldCIST 2024 (World Conference on Information Systems and Technologies), edited by Á. Rocha, Springer Nature Switzerland AG, 2024, pp. 66–75. https://doi.org/10.1007/978-3-031-60227-6_6.Copy MLA citation

IEEE

N. R. Vajjhala and K. D. Strang, “An Exploratory Big Data Approach to Understanding Commitment in Projects,” in WorldCIST 2024 (World Conference on Information Systems and Technologies), Á. Rocha, Ed. Springer Nature Switzerland AG, 2024, pp. 66–75, doi: 10.1007/978-3-031-60227-6_6.Copy IEEE citation

@inproceedings{vajjhala2024machine,
  title = {An Exploratory Big Data Approach to Understanding Commitment in Projects},
  author = {Vajjhala, Narasimha Rao and Strang, Kenneth David},
  booktitle = {WorldCIST 2024 (World Conference on Information Systems and Technologies)},
  series = {Lecture Notes in Networks and Systems},
  editor = {Rocha, Á.},
  pages = {66--75},
  year = {2024},
  publisher = {Springer Nature Switzerland AG},
  doi = {10.1007/978-3-031-60227-6_6},
  url = {https://doi.org/10.1007/978-3-031-60227-6_6}
}

TY  - CPAPER
TI  - An Exploratory Big Data Approach to Understanding Commitment in Projects
AU  - Vajjhala, Narasimha Rao
AU  - Strang, Kenneth David
ED  - Rocha, Á.
T2  - WorldCIST 2024 (World Conference on Information Systems and Technologies)
T3  - Lecture Notes in Networks and Systems
PY  - 2024
DA  - 2024/05/16
VL  - 989
SP  - 66
EP  - 75
PB  - Springer Nature Switzerland AG
DO  - 10.1007/978-3-031-60227-6_6
UR  - https://www.narasimharao.net/research/machine-learning-project-manager-organizational-commitment-big-data/
LA  - en
ER  - 

Links and files

• DOI: https://doi.org/10.1007/978-3-031-60227-6_6
• This page as text: https://www.narasimharao.net/research/machine-learning-project-manager-organizational-commitment-big-data/paper.md
• Permanent address of this page: https://www.narasimharao.net/research/machine-learning-project-manager-organizational-commitment-big-data/