CMS Is Rethinking How It Measures AI Success — From Usage Numbers to Real Outcomes

What does it mean for a large organization to successfully adopt AI? For the Centers for Medicare and Medicaid Services, the answer is shifting from "how many people use it" to "what did it actually accomplish."

CMS Chief Information Officer Patrick Newbold recently discussed how the agency is moving away from tracking raw AI adoption metrics — like the percentage of employees using AI tools — toward measuring the concrete outcomes those tools produce. With approximately 80 percent of its workforce already interacting with AI in some capacity, including through its internal assistant CMS Chat, CMS has reached a point where usage alone is no longer a meaningful signal of progress.

Key Takeaways

• The metric shift: CMS is pivoting from measuring AI usage to measuring AI outcomes — a model Newbold describes as starting with the result you want and then asking whether AI is the right tool to get there
• Current AI footprint: CMS's AI portfolio includes more than 70 documented use cases from the Department of Health and Human Services' 2025 AI use case inventory, alongside department-wide generative AI tools available through federal contracting vehicles
• What this signals: As federal agencies mature in their AI deployments, outcome-based measurement is becoming a more credible benchmark than adoption rates alone — a pattern likely to influence how both public and private sector organizations evaluate AI program performance

For organizations still focused on AI adoption dashboards and usage percentages, CMS's approach offers a reframe: the question is not how many people touched AI this month, but what changed because of it.

Read the full article on ExecutiveGov