Research synthesis · ESG & sustainability measurement

ESG compliance measurement: why ratings and records mislead, and what can be measured reliably

Strang and Vajjhala show that ESG compliance can be measured reliably at project level with a validated seven-item instrument, but that firm-level ESG ratings, organizational evaluation records and supply chain risk perceptions are much weaker measures than they appear.

A short narrative review of research by Narasimha Rao Vajjhala and co-authors in this area. Every claim carries an APA citation that links to the publication’s own page (abstract, key findings, DOI); the full references are listed at the end. Updated 4 October 2026.

Synthesis

In a connected series of studies, Kenneth David Strang and Narasimha Rao Vajjhala have examined whether Environmental, Social, and Governance (ESG) compliance can be measured reliably, and at which level. Their starting point is that ESG frameworks measure firms and the Sustainable Development Goals measure nations, whereas sustainability commitments are carried out in projects and supply chains (Strang & Vajjhala, 2026c).

At the firm level, the evidence is sobering. For 14 asset management firms each holding more than USD 1 trillion, Strang and Vajjhala (2024a) found that ESG scores from MSCI, Sustainalytics and CSRhub were unrelated to the misconduct and arbitration cases decided against the firms (all Bayes factors below 1), and that the providers disagreed with one another (MSCI versus Sustainalytics r = −0.069). The ratings studied therefore do not capture the anti-corruption evidence that governance scores are meant to reflect.

At the project level, the picture is more encouraging. Surveying 2,231 project sponsors and decision-makers in North American manufacturing, Strang and Vajjhala (2026c) reduced a 30-item, six-dimension draft to a validated seven-item, two-factor instrument — ESG planning, and ESG monitoring and controlling — with excellent fit (CFI = 0.99, TLI = 0.98, RMSEA = 0.052). The instrument gives project offices a standardized benchmark that can be administered at close-out.

Organizational records, however, limit what machine learning can learn. In 207 project records from a financial software engineering firm, Strang and Vajjhala (2026a) found that stakeholder-rated social and governance factors tracked the overall project score almost perfectly (r = +0.995 and +0.966) while the environmental factor was unrelated to it, and that classifiers performed at or near chance (kNN AUC = 0.497). In 1,988 supply chain decision records, labor and environmental risks were rated lowest of six regulatory-volatility risks (medians 0 and 1 on 0–5 scales) during a period of record forced-labor enforcement, and an apparently perfect classifier reflected label leakage rather than predictive skill (Strang & Vajjhala, 2026b).

The combined conclusion is that how ESG data are generated shapes what can be learned from them. Strang and Vajjhala recommend process-based project instruments, specific disclosure of legal proceedings, multidimensional rather than composite risk dashboards, and provenance metadata and leakage diagnostics for AI-enabled ESG tools.

Key claims with sources

ESG ratings from MSCI, Sustainalytics and CSRhub were unrelated to the misconduct and arbitration records of 14 trillion-dollar asset managers (Strang & Vajjhala, 2024a).
A seven-item, two-factor instrument measures project-level ESG compliance with excellent fit (CFI = 0.99, RMSEA = 0.052) (Strang & Vajjhala, 2026c).
Supply chain decision-makers rated labor and environmental risks lowest of six regulatory risks during record forced-labor enforcement (Strang & Vajjhala, 2026b).

For a longer review that sets these studies against the wider literature, see the evidence overview Can ESG compliance be measured reliably? Evidence from ESG ratings, project-level instruments, machine learning and supply chain records

The publications and what each contributes

Publications cited in this synthesis, with the method and core finding of each
Publication	Method	Core finding
Strang and Vajjhala (2026b) Sustainability	Quantitative retrospective analysis of archival organisational records (N = 1,988): descriptive statistics, exploratory factor analysis with parallel analysis, and binary logistic regression with classification diagnostics.	Perceived regulatory-volatility risks do not form a single construct (KMO = 0.470), and labor and environmental risks were rated lowest of all (medians 0 and 1 on 0–5 scales) during a period of record forced-labor enforcement.
Strang and Vajjhala (2026a) Systems	Single-case socio-technical systems study of archival project records (N = 207) using an open-source data science workflow: correlation analysis, principal component analysis, common-method-bias diagnostics, and kNN and SVM classification.	In 207 financial software project records, stakeholder-rated social and governance factors tracked the overall project score almost perfectly (r = +0.995 and +0.966) while the environmental factor was unrelated to it, and machine learning classifiers performed weakly (kNN AUC = 0.497; SVM accuracy 61.8%).
Strang and Vajjhala (2026c) Information	Quantitative instrument development and validation with survey data from 2,231 project sponsors and decision-makers in North American manufacturing (NAICS 31–33): principal component analysis, exploratory factor analysis and structural equation modelling.	A survey of 2,231 North American manufacturing project sponsors produced a validated seven-item, two-factor instrument — ESG planning and ESG monitoring and controlling — with excellent fit (CFI = 0.99, TLI = 0.98, RMSEA = 0.052, SRMR < 0.035).
Strang and Vajjhala (2024a) Sustainability	Quantitative archival study of 14 large asset management firms combining AI-assisted collection of legal, misconduct and arbitration records with ESG ratings from three providers, analysed with Bayesian correlation and bootstrapping.	For 14 asset managers, each with USD 1 trillion or more in assets under management, no ESG score from MSCI, Sustainalytics or CSRhub was significantly correlated with misconduct or arbitration case counts, and MSCI and Sustainalytics scores for the same firms were uncorrelated (r = −0.069).

Key terms in this area

GRI 206-1: The Global Reporting Initiative disclosure on legal actions for anti-competitive behavior, anti-trust, and monopoly practices — used here as an anti-corruption governance indicator. — as used in Strang and Vajjhala (2024a)
ESG rating divergence: When different rating agencies give the same company substantially different ESG scores. — as used in Strang and Vajjhala (2024a)
Bayes factor (BF): The ratio of evidence for one hypothesis over another; values above 100 are conventionally treated as extreme evidence. — as used in Strang and Vajjhala (2024a)
Project-level ESG compliance: The extent to which Environmental, Social, and Governance requirements are planned for and then monitored and controlled within a specific project, as opposed to being reported only at company level. — as used in Strang and Vajjhala (2026c)
Project sponsor: The senior person or group who authorizes a project, provides its funding, and is accountable for its business outcome. — as used in Strang and Vajjhala (2026c)
Model fit indices: CFI and TLI near 1 and RMSEA and SRMR near 0 indicate that a measurement model reproduces the observed survey data well. — as used in Strang and Vajjhala (2026c)
Regulatory volatility: Rapid, hard-to-predict change in the rules that govern trade and operations — tariffs, customs regimes, labor and environmental (ESG) regulation — that makes supply chains less predictable. — as used in Strang and Vajjhala (2026b)
Sustainability leakage: As used in this study: when surprise ESG enforcement leads firms to exit suppliers or move sourcing elsewhere instead of remediating conditions in the scrutinized region. — as used in Strang and Vajjhala (2026b)
Label leakage: When a predictor variable effectively encodes the outcome being predicted, producing unrealistically perfect model accuracy that will not hold in real use. — as used in Strang and Vajjhala (2026b)

All key terms across the publications →

Standards, datasets and methods named in these studies

United Nations Sustainable Development Goals International framework
United Nations. The 17 Goals — Sustainable Development Goals.
Named in Strang and Vajjhala (2026c)
North American Industry Classification System (NAICS) Classification standard
U.S. Census Bureau. North American Industry Classification System (NAICS).
Named in Strang and Vajjhala (2026c)

References

Strang, K. D., & Vajjhala, N. R. (2024a). Evaluating the Anti-Corruption Factor in Environmental, Social, and Governance Indices by Sampling Large Financial Asset Management Firms. Sustainability, 16(23), Article 10240. https://doi.org/10.3390/su162310240 Summary & key findings →
Strang, K. D., & Vajjhala, N. R. (2026a). Machine Learning and Data Science for ESG Compliance Measurement in Financial Software Engineering Projects: A Single-Case Socio-Technical Systems Analysis. Systems, 14(8), Article 990. https://doi.org/10.3390/systems14080990 Summary & key findings →
Strang, K. D., & Vajjhala, N. R. (2026b). Regulatory Volatility in Digital Supply Chains: An Information Systems Analytics Study of Decision-Maker Risk Perceptions and Sustainability-Related Outcomes. Sustainability, 18(18), Article 9359. https://doi.org/10.3390/su18189359 Summary & key findings →
Strang, K. D., & Vajjhala, N. R. (2026c). Verifying SDG ESG Compliance in Manufacturing Industry Projects by Surveying Sponsors. Information, 17(4), Article 311. https://doi.org/10.3390/info17040311 Summary & key findings →

How to cite this synthesis

Please cite the original publications above for specific findings. To cite this overview itself:

Vajjhala, N. R. (2026, October 4). ESG compliance measurement: why ratings and records mislead, and what can be measured reliably. Narasimha Rao Vajjhala. https://www.narasimharao.net/research/focus-areas/esg-sustainability-measurement/

Markdown version