AI Models Score Well on Clear HR Tasks but Fail at Nuance, New Benchmark Study Finds

A new benchmark study from PYX Labs, a research lab sponsored by Perceptyx, finds that AI models handle straightforward HR tasks reliably but break down when asked to synthesize open-ended employee feedback—raising concerns for companies that have integrated AI into performance management workflows.

The study tested seven AI models from OpenAI, Google, Anthropic, and xAI across 84 employee listening tasks, measuring results against criteria developed by psychologists and organizational behavior specialists.

Key Findings

• AI models passed 76%–82% of tasks with clear, verifiable answers
• For tasks requiring synthesis and interpretation of nuanced employee feedback, scores dropped to as low as 33%
• Synthesis was the lowest-scoring capability across every model tested, ranging from 14% to 57%
• Researchers found rare but meaningful instances where models produced fabricated statistical outputs or failed to stay within dataset constraints
• 37% of companies currently use AI tools as part of their performance management process (WTW, 2025), despite only 20% saying their managers are effective at coaching and feedback

Joseph Freed, Chief Product Officer at Perceptyx, noted that the issue is not whether AI can produce fluent answers—it's whether AI understands what "good" looks like in a workplace context. The study concluded that AI models are unreliable without human oversight when it comes to interpreting employee feedback.

🔗 Read the full article on HR Dive