Why AI Strategy Fails: The Average Customer Assumption CDOs Must Fix

Why Enterprise AI Keeps Missing the Mark: The Average Customer Problem

IBM puts average enterprise AI ROI at roughly 6% — below most organizations' cost of capital. MIT and BCG found about 95% of enterprise generative AI pilots produced no measurable P&L impact. Michael Podgortsev, Director of Data and AI with 15 years of enterprise experience, argues the standard explanations (data quality, talent gaps, change management) are symptoms of a more fundamental flaw upstream.

The Core Problem

Foundation models return the most probable response given their training distribution. For inputs in the dense center of that distribution, they perform well. For inputs in sparse regions — customers who differ from the majority the model was trained on — the model does not fail loudly. It returns an equally fluent, equally confident response drawn from the nearest pattern it knows. Standard monitoring layers are not designed to detect this.

This means a model can clear every vendor benchmark in procurement and still be systematically unreliable for a segment central to your business.

A Real Example

At a financial services firm, an AI-assisted underwriting tool cleared every vendor benchmark and sailed through procurement. A year in, dashboards showed green metrics across accuracy, satisfaction, and throughput — but one regional team kept escalating cases by hand. When the data was segmented, the cause was clear: that team's customers were largely self-employed and multi-income, and the tool's implicit employed/unemployed model handled them poorly. It had been confidently wrong for that segment for a year.

What CDOs Can Do About It

• In vendor due diligence: Ask how the model's training data represents your specific customer population and what segment-level evaluations exist beyond standard benchmarks.
• In pilot governance: Require segmented performance by customer dimension as a formal go/no-go criterion, not aggregate accuracy alone.
• In ongoing monitoring: Add segment-level drift tracking so a model that starts well cannot quietly degrade for one population after it scales.

Read the full article on CDO Magazine