Your AI Center of Excellence Will Fail If IT Owns It Alone

There’s a SharePoint site for it. There’s a Teams channel. A handful of architects are involved, a Copilot admin or two, maybe a data engineer who got “voluntold” into the role. The AI Center of Excellence (CoE) is officially stood up, and IT is running it.

Six months in, it’s producing documentation nobody outside of IT is reading, approving use cases the business didn’t ask for, and issuing governance decisions that business units are actively working around. The sales team built their own Power Automate agents. HR found a third-party summarization tool. Everyone is perfectly polite about it in meetings.

This is not a technology failure. It’s a seating chart failure — and it happens consistently enough to qualify as industry-standard practice.

What a CoE Is Actually Supposed to Do

An AI CoE is not a team that builds AI things. It’s not a request queue. It is a cross-functional capability that connects AI strategy to business outcomes, sets standards that other teams build against, and owns the governance model that keeps the whole thing from producing liability faster than it produces value.

That definition has a structural implication most organizations miss: the people who need genuine decision-making authority in a CoE are not primarily technical. They’re the people who understand where AI creates real business value, who own the data, who carry legal and compliance accountability, and who will actually be responsible for whether employees adopt any of this.

IT is one of those groups. Not the group.

Why IT-Led CoEs Stall

When IT owns the CoE, business stakeholders experience it as a checkpoint rather than a resource. The governance framework is technically sound. The sensitivity label schema is defensible. And then the business routes around it, because the approval process takes three weeks, nobody explained why it exists, and the workaround is two clicks in Power Platform.

That’s not rebelliousness. That’s rational behavior in response to a process that wasn’t built with them in mind.

There’s also a use-case prioritization problem. IT evaluates technical feasibility and risk very well. It is generally not positioned to evaluate business value. A CoE that prioritizes initiatives by technical complexity will consistently fund the wrong projects — and the business will eventually stop submitting requests and start building things quietly. If that sounds familiar from the last article on agent sprawl, it should. The same dynamic that produces shadow IT produces shadow agents.

The Microsoft Lens

Microsoft Digital, the company’s internal IT organization, did not build an IT-owned CoE when it deployed Copilot and began governing agents at scale. It built an AI council structure: cross-functional, with accountability spanning IT, security, business stakeholders, and senior leadership. The reasoning was direct: AI initiatives without cross-functional alignment produce technical output without measurable business outcomes. That is not a controversial finding. It is just rarely acted on.

Microsoft’s Copilot Studio agentic maturity model reflects this same structure. At the highest maturity levels, a cross-functional AI Council provides active oversight and escalation for high-impact cases. Responsible AI standards get translated into concrete practices, not handed down as policy from IT and hoped for the best. Policy distributed is policy ignored. However, standards built collaboratively get followed. If you’ve ever listened to my podcast, one of my most often repeated “Buckleyisms” is

“The more you involve people in the process, the more likely those people will follow the process.”

What Cross-Disciplinary Actually Means

I spent a good chunk of my career running project management teams and owning various councils and committees, and am a big fan of the CoE model. In my experience, a functioning AI CoE for a Microsoft 365 environment needs representation from at least five areas, each with a real seat rather than an advisory one.

IT and Security own the technical standards, the agent registry, sensitivity label schemas, and the Entra identity model for human and non-human identities alike. Most organizations already have this constituency. It’s the others that are missing.

Business Unit Leadership brings the use case pipeline and the domain knowledge to evaluate it. They know where AI creates actual productivity gains in their function, and they have the organizational credibility to drive adoption when a deployment lands on real employees’ workflows.

Legal and Compliance owns the regulatory exposure map — what Purview is enforcing, where DLP policies apply, what sector regulations require, and what the liability picture looks like when an autonomous agent causes harm. This seat is not optional.

HR and Change Management is the function most consistently absent from CoE structures, and that absence is the most direct explanation for flat adoption numbers. Copilot moving from licensed to actively used requires training design, behavior change, and manager reinforcement. None of that materializes without HR at the table.

Finance closes the loop. If the CoE cannot connect AI investments to measurable outcomes, it loses executive support. Finance defines what “working” looks like before deployment, not after someone needs to justify the spend.

The Structure That Scales

The pattern that actually works in complex enterprises is hub-and-spoke: a small central CoE owns standards, governance, and the platform layer, while embedded AI leads in each business unit own local deployment and use case development within those standards. Maybe it’s owned by your Project Management Organization (PMO). Maybe it’s a shared services team. Or maybe it is truly a cross-organizational team, with representatives from across the org. Whatever makes sense within your company.

The hub does not approve every agent. It approves the framework within which agents get built. Business units move at their own speed inside it. The CoE becomes a standards body and escalation path — not the bottleneck that everyone has learned to route around.

This also addresses the agent ownership problem. Agents become ownerless when their creator changes roles or leaves. The right answer isn’t just a detection workflow in the Agent 365 admin center — though Microsoft has built one. It’s ensuring every agent has a business owner, not just an IT owner, built into the governance model from day one.

The Question Worth Asking Now

Before formalizing your AI CoE — or auditing the one already running — the useful question is not “who should lead this?” It’s “who has a stake in the outcomes, and do they actually have a seat?”

If the honest answer is “mostly IT,” you have a body that will produce technically excellent work the business routes around. A cross-disciplinary CoE doesn’t slow things down. It eliminates the friction that makes the workaround feel faster than the process.

That’s the difference between a CoE that produces governance documents and one that produces results.