How to Evaluate an AI Consultant (Without Getting Burned)

Most companies don't get burned by bad AI consultants.

They get burned by not knowing what they needed before the consultant showed up.

That's a clarity failure, not a consultant failure. A smart person walks in, asks "what do you want?", and suddenly you have a month of wasted cycles before anyone's built anything.

Here's the exercise I'd do before hiring anyone: walk your organization and map the processes that are the most human-intensive. Ask yourself — if every department only had one person to do all the work of that department, what would start breaking? What would pile up?

Those bottlenecks are the only thing worth handing to a consultant. Go in with that list and the engagement starts immediately. Without it, you're paying someone to figure out what you should have figured out yourself.

---

The credential that actually matters

The credential that matters is whether they've had an operational role close to business process, not their AI certifications.

People who've worked through operations understand pipelines, bottlenecks, quality control, data hygiene. They know when AI makes sense to insert into a process. They know how to set up AI so the outputs are actually usable. People from a purely academic background often have the theory right and miss where the real complexity lives — in the messy handoffs between systems and humans that don't appear in any benchmark.

Academic credentials look impressive but operational experience is what the work requires.

---

The red flag hiding in plain sight

Frameworks aren't inherently bad. Having a way to organize a plan is useful.

The red flag is when a consultant leads with jargon — the Five Pillars of Enterprise AI, the AI Maturity Model — and the frameworks are more for show than for substance. Ask them to walk through how they'd approach a specific problem at your company. The answer should get specific fast. If it stays at the level of concepts, they're selling the framework.

---

The interview question that reveals judgment

Ask: "What are problems AI is not ready to solve today?"

It flips the usual dynamic. Instead of staying in the land of AI-as-panacea, it forces them to show judgment about where AI genuinely falls short. The hype moves faster than the field — anyone worth hiring should be able to name real limitations with real specificity.

No good answer is a red flag; a vague one without examples is a yellow flag. A sharp, specific answer that shows they've hit those limits firsthand is what you're looking for.

---

Name recognition

Podcast appearances, LinkedIn following, Forbes quotes — none of it is a positive or negative signal. Ignore it and focus on what they've shipped and what they understand.

---

Define done before you start

Set measurable outcomes before the engagement begins. Not vague goals — actual metrics that tell you whether you're making progress within the contract length.

Start with one specific, human-heavy process, define a clear outcome, and work on that. Add new problem spaces at the end if it's going well. Companies that hand a consultant all their problems at once get diffuse progress on everything and real progress on nothing.

---

What you're actually looking for

Credentials and frameworks are proxies. The real thing you're looking for is a first-principles thinker with genuine curiosity — someone who keeps asking why things are done a certain way and wants to understand what the business is actually trying to achieve before suggesting anything.

That kind of person, pointed at a well-scoped problem with a clear success metric, is what a good AI consulting engagement looks like.

The filter is clarity — yours going in, and theirs once they start asking questions.