AI coaches promise relief for overloaded managers but how should HR measure them?

ADP's playbook for proving an AI coach: Implement it on moments HR already measures, then judge adoption depth and outcomes

AI coaches promise relief for overloaded managers but how should HR measure them?

Employers are handing overloaded managers an artificial intelligence (AI) coach, and HR now has to show the tool earns its renewal.

ADP, which deploys Valence's AI coach Nadia internally, has logged about 140,000 coaching conversations since launch – nearly 3,000 a week – according to Chief Talent Officer Jay Caldwell. ADP's venture arm invested in Valence in July 2026, ADP announced in its July 2026 release, and ADP sells HR software, so its experience is that of a buyer with a stake in the category.

"We actually do use Valence or Nadia as the AI coach that we use after testing many," said Caldwell, chief talent officer at Roseland, New Jersey-based ADP. "Our HR team stays very closely connected with our business team on these types of partnerships."

How to scope an AI coach narrowly enough to test

ADP started with three use cases, Caldwell said: on-demand help for managers handling "a sticky people situation," support for formal development programs and help at key talent moments. "Could AI actually help scale the support HR provides to our managers on demand," he said. "Sounds really cool, but like one of the questions was, would it stick and would people actually leverage it?"

The talent-moment nudges worked best. When ADP's quarterly engagement survey shows a team's scores dropping, the manager gets a prompt to prepare with the AI coach. "Rather than the managers just kind of like dwelling on that data by themselves... we're giving them help in that moment," Caldwell said. ADP repeats the nudge before year-end compensation conversations.

Why a purpose-built coach differs from a chatbot

Caldwell said the value lies in customization and in coaching rather than advice. ADP trained the coach on its six-capability "values-driven leadership" framework, including building trust and belonging. "We've actually taught it how we think about leadership at ADP and how we want our managers to manage that moment," he said.

"A coach will push you to insights on your own rather than just giving the answers," Caldwell said. A general-purpose AI tool, he argued, tends to affirm whatever the user already thinks. "That's typically what AI tools do is encourage whatever's in your mind and tell you you're brilliant," he said.

Caldwell said ADP Research ties team engagement closely to trust in the team leader, and the coach's conversation themes reflect that: trust, collaboration and cross-cultural management. One engagement item ADP tracks – "I believe my team members have my back" – gives managers a specific trust signal to work on with the coach.

Experience, adoption, impact

Caldwell said ADP tracks three layers. The first is experience, gathered through manager conversations and a net promoter-style question: "Would you recommend this tool to others in the organization?" The second is adoption: "how many leaders are actually leveraging the tool, how consistently are they leveraging it, and how deep do they go." The third is impact on outcomes such as engagement, performance and turnover.

Impact is the hardest to isolate at a company already scoring well. "In terms of our engagement, performance, turnover, all these metrics and outcomes are very... we're in a great place as an organization. It would be easier to measure if we had lots of problems," Caldwell said.

Written into renewal criteria, that sequence – experience, then adoption depth, then team-level outcomes tied to triggered moments – gives a defensible path that doesn't rely on a vendor's usage dashboard alone.

Augment, don't replace, existing coaching

ADP built its business case on reach, not efficiency. "How could we possibly resource a professional coach for every single leader?" Caldwell said. ADP has since opened the coach company-wide. "Would have never been possible to do that for 67,000 people," he said. The tool sits alongside ADP's leadership programs rather than replacing them.

Guardrails matter as much as features. Caldwell urged buyers to vet a partner's privacy and security, since coaching conversations carry sensitive relationship information, and said confidentiality drives candor. ADP also routes some topics away from the tool. "If you start talking about something that you really should be engaging your HR partner, like an employment decision as an example, that's not a topic for an AI coach," he said. "You are talking to an AI. You're dealing with humans. And so you own your decisions."

LATEST NEWS