How to evaluate an AI governance partner before you sign anything

Every consultancy says it does "responsible AI" now. The words on the homepage rarely tell you anything - what matters is whether the answers hold up when you ask specific questions before signing. Here's what to actually ask.

Ask who owns the decision when the model is wrong

A vague answer here ("the AI flags it for review") is the single biggest tell. A real governance partner can name the exact human-in-the-loop escalation path: what triggers a hold, who reviews it, what the fallback is when that person is unavailable, and how long a decision can sit before it times out. If they can't draw that flow on a whiteboard in under two minutes, they haven't built it before.

Ask for the audit trail, not a description of one

"We log everything" is not an audit trail. Ask to see one: a real record of a real decision, what data it drew from, what the model returned, who reviewed it, and when. If what they show you is a generic architecture diagram instead of an actual log entry, the audit trail exists on a slide, not in production.

Ask what happens to your data after the engagement ends

This is where a lot of AI vendors go quiet. Does your data train a shared model that benefits their other clients? Is it retained after offboarding, and under what policy? A partner that's actually built for regulated industries has a clean, specific answer, not "we'll figure that out in the contract."

Ask for one reference who said no to something

Every vendor has references who'll say the engagement went well. Ask instead for a reference who can describe a time the vendor pushed back on a request, flagged a risk, or refused to ship something that wasn't ready. A partner that's never said no to a client hasn't been tested yet.

• Who owns the decision when the model is wrong, specifically
• A real audit trail entry, not a diagram of one
• A written answer on data retention and reuse after offboarding
• A reference who can describe a time they said no

None of this is exotic. It's just specific, and specificity is what a slide deck can't fake.