10 Questions to Ask Before You Hire an AI Consultant

The AI consulting market has a quality distribution problem. On one end: practitioners who have built real systems, have production scars, and can tell you exactly where your project is likely to fail. On the other end: people who’ve read the same blog posts you have, took a three-week course, and are now billing at $200/hour to fumble through workflows they’ve never shipped.

The gap between those two types of consultant is enormous. The gap in their marketing is nearly invisible.

I say this as someone on the practitioner side of that equation — not to self-promote, but because I’ve watched clients spend real money on people who were figuring it out on the client’s dime. You deserve the information to tell the difference before you write a check. So here are the questions.

1. “Can you walk me through a production AI system you’ve built — what it does, what broke, and how you fixed it?”

This is the single most useful question on the list. A consultant who has actually shipped AI in production can answer it immediately and in detail. They know what broke because they were the one who got paged at 11pm when the pipeline choked on malformed input. They can tell you which failure mode was the most embarrassing and what the fix actually looked like in code.

A consultant who is mostly theoretical will give you a high-level description of a system. Vague verb tenses. No specific failures. No debugging stories. The failures are where the real knowledge lives — if they can’t talk about them, they probably didn’t have them.

2. “What does your typical engagement look like in week one?”

The wrong answer is discovery workshops, stakeholder alignment sessions, and requirements documentation. Those things have value in the right context, but leading with them signals a process more interested in billing discovery than in understanding what you actually need.

The right answer — or at least a reassuring one — is something about seeing your data early, getting access to your systems, running a quick proof of concept, or at minimum defining what a measurable win in 30 days would look like. Consultants who’ve shipped things start thinking about shipping things immediately. Those who haven’t default to process.

3. “How do you price your work, and what does success look like financially?”

Watch out for pure hourly billing with no outcome orientation. Hourly billing creates a structural misalignment: the consultant benefits from work taking longer, and you bear the entire risk of scope expansion. It’s not that hourly is always wrong — it’s that hourly with no defined scope and no outcome milestones is a blank check.

Better arrangements include fixed-price scopes with clear deliverables, milestone-based payments, or hourly within a capped scope. The consultant should be able to tell you what you’ll have at the end of the engagement and what it will cost. If the answer to “what does this cost” is “it depends on how long it takes,” keep asking until you get a number with a ceiling.

4. “Who owns the code and infrastructure at the end?”

This question surfaces a business model decision that most consultants don’t volunteer. Some consultants build on proprietary platforms or tools they license — which means your AI system has a vendor dependency that the consultant controls. When the engagement ends, the ongoing cost structure stays, and so does your dependency.

The answer you want: you own everything. The code lives in your repository. The infrastructure runs on accounts you control. The consultant hands you documentation and walks away — or stays for retained support at your discretion, not by necessity.

This directly connects to the broader question of client-owned AI infrastructure versus AI SaaS. The choice isn’t just about platforms — it starts with who your consultant builds for: you or themselves.

5. “What AI use case have you seen fail most often, and why?”

Good consultants have strong, specific opinions about failure modes because they’ve watched them play out. They can tell you exactly how the “we’ll use AI to replace our entire customer service team” initiative fails, or why the “let’s build a chatbot on our knowledge base” project consistently underdelivers its first version. These are hard-won opinions, not recycled LinkedIn content.

If the answer is vague or optimistic — “all use cases can work with the right implementation” — that’s a yellow flag. The right implementation still has to reckon with real constraints. Experience produces opinions about failure. The absence of opinions suggests the absence of experience.

6. “How do you handle a project where the AI output quality plateau’s and can’t improve further?”

This is a question about data quality problem-solving. Almost every AI project hits a ceiling where improving the model, prompt, or architecture doesn’t help — because the ceiling is the quality of the training or input data. How a consultant handles that moment tells you a lot.

The right answer: dig into the data. Profile it. Identify structural inconsistency, label disagreement, temporal drift. Fix the data before continuing to iterate on the model. (This is the path I’ve written about in detail.)

The wrong answer: keep tuning hyperparameters, try a different model, add more examples. Treating a data problem as a model problem is one of the most expensive mistakes in AI development, and a consultant who doesn’t instinctively go to the data when quality stalls probably hasn’t seen that failure mode enough times to recognize it.

7. “What does your handoff look like?”

A good engagement ends with you in control of a system you can operate and maintain without the consultant. That means documentation written for your team, not for someone with a PhD in ML. It means infrastructure you can SSH into, modify, and hand to another engineer if needed. It means a codebase with commit history that explains why decisions were made.

Ask specifically: “If I had to replace you with a different engineer six months after we wrapped up, how hard would that be?” The answer should be “not very hard” with a specific reason why. If the answer involves proprietary tools, undocumented choices, or tribal knowledge that lives in the consultant’s head — that’s leverage they’re building over you, intentionally or not.

8. “Can you share an example of a scope you pushed back on, and why?”

Consultants who optimize for billing say yes to scope. Consultants who optimize for outcomes push back when the scope is wrong.

A good answer sounds like: “A client wanted to build a multi-agent system for their first AI project, and I talked them out of it — the simpler pipeline would have solved 90% of the problem with 20% of the complexity. We built the simple version first.” Or: “A client wanted to integrate five different AI tools simultaneously, and I said we should pick one, prove the value, and expand from there.”

The willingness to recommend less — to simplify, to descope, to say “you don’t need that yet” — is a signal that the consultant’s interest is aligned with yours, not with maximizing their own invoice.

9. “How do you think about AI guardrails and human oversight?”

I’ve written extensively about why full AI automation is a mistake and how to build guardrails into agentic systems. This is an area where the difference between a naive and experienced practitioner is stark.

A naive answer: “We’ll add monitoring once the system is stable.” A thoughtful answer: “Logging, retry logic, and human approval checkpoints are built into the architecture from day one — not bolted on later. We define what actions require human approval before we write code for them.”

AI systems that fail silently or take unreviewed irreversible actions are operational liabilities. The consultant should have a clear opinion about how to prevent that — and that opinion should be part of their standard approach, not something you have to ask for specifically.

10. “Why wouldn’t this project work?”

The best consultants are the easiest to get a straight answer from on this question. They’ve seen projects fail. They know the risk surface. They can tell you, without hedging, exactly what could go wrong and what they’d do about it.

Bad data quality. Unclear success criteria. Stakeholder alignment problems. Unrealistic timelines. Regulatory exposure for your specific industry or use case. Organizational resistance to the process change that’s required for AI to deliver value. These are real risks — the experienced consultant has seen all of them up close.

If the answer is sunny and vague — “every project has challenges but we’ll work through them together” — that’s not due diligence. That’s sales language. You want someone who can look at your specific situation and tell you specifically what might break.

---

The right AI consultant for your business makes you better at operating your business. They hand over something that works, that you understand, and that you own. They are thoroughly, intentionally unnecessary by the time they leave.

Ask these questions. Listen to the specificity (or lack of it) in the answers. The right person will be easy to identify — not because they have the smoothest pitch, but because they can talk about failure without flinching.

If you’d like to run through these questions with me — or if you just want to talk through what your AI project actually needs — you know where to find me.