12 Questions to Ask an AI Agent Vendor Before They Say “Two Weeks”

1986. Tom Hanks and Shelley Long buy a mansion for $200,000.

It’s a steal. It’s also falling apart. The door frame comes out of the wall. The staircase collapses.

Every time anyone asks the contractors how long, the answer is the same.

“Two weeks.”

It takes a lot longer.

Anyone who’s renovated a kitchen has met that crew. The questions to ask an AI agent vendor are mostly the ones you’d ask them.

I run a firm that builds AI agents for companies of 50 to 500 people, so I usually sit on the other side of this table. These are the questions I’d ask as the buyer, including of us.

In June 2025, Gartner gave part of the problem a name, “agent washing”: chatbots, assistants and RPA tools relabeled as agents. It estimated only about 130 of the thousands of agentic AI vendors are real.

A good vendor answers each of the twelve below with a price, a name, a date or a rule. Anything vaguer is a “two weeks.”

What should you ask about the price?

1. Is the price fixed, or is there a fee every month?

You’re either paying for a finished job or paying every month, and both are legitimate. Either way, find out what the number assumes (how many jobs, how many systems, who signs off) and what it becomes at twice today’s volume.

2. What does month 13 cost?

Everyone prices month one. After launch an agent still needs hosting, AI usage and somebody to adjust it when a system underneath changes. Get that in writing, at your volume, split by who gets paid and marked where it’s optional. How Much Does an AI Agent Cost? breaks it down.

3. What happens if it can’t do the job?

Some jobs turn out wrong for an agent, usually because the rules live in Linda’s head and nobody has written them down. (Linda knows which supplier always rounds up.) Ask where you’ll both find out, and what you owe by then. We bill half at kickoff and half at go-live, and if testing on your own past work shows it can’t do the job, we stop and you don’t pay the second half.

Who owns it, and who builds it?

4. Who owns the code, the accounts and the data?

Your name should be on the cloud account, the AI account and the code. Ask what each step sends to the AI provider, and whether the provider can train on it under your account’s terms. The agent should get its own login with only the permissions the job needs, so connecting it to HR doesn’t open payroll. Renting is fine if you choose it. “It runs on our infrastructure” with no way out is not, and here’s what changes when you own the software instead of renting it.

5. Who is on the team, by name?

Somebody has to own the project, somebody has to understand how your process and systems fit together, and somebody has to write the code. On a small job one person can cover two of those. One brilliant freelancer is still one person, though. If the person who sold it to you is the only name you ever get, keep asking, and find out which of them you’ll talk to every week.

How do you know it’s a real AI agent?

6. Can you walk one of our jobs from arrival to done?

Pick one job and have them walk it: what it reads, checks and writes, and where it stops. Then ask who you can call that’s had one running for six months. Gartner makes a fair point here: “Many use cases positioned as agentic today don’t require agentic implementations.” If a rule in Zapier would do the job, that’s the cheaper tool, and a good vendor will say so.

7. Will you test it on our own past work?

Demos use clean examples. Real invoices come in crooked scans, with three POs on one page, from a supplier who still faxes. Past cases where you already know the right answer are the test, and you should see every one it got wrong before anything goes live.

8. What does it do when it isn’t sure?

The case an agent can’t handle matters more than the hundred it can.

It should stop, flag the case, say why, and hand it to a named person with what they need to decide. “It’s 99% accurate” with no plan for the other 1% is a sales line.

9. Who signs off exceptions, money and customer messages?

“Fully autonomous” sounds great until the thing is moving money or writing to a customer. Expect a role on your team for each. On an invoice matching agent, a line $212 over the purchase order waits for the controller, with the invoice, PO and receipt side by side.

How long does it take to get an AI agent live?

10. How many weeks, and what do you need from us?

In my experience the slow part is access, not code, like a login request sitting in IT’s queue. Your team’s hours go on system access, past examples, someone who signs off, and an hour a week from a person who does the job today. Ask who tells the people whose job changes, and when. That part decides whether it sticks (our workflow first explainer is about exactly this). Doubt any timeline that asks nothing of you.

How will you know it worked?

11. What number will you measure, and when?

Name one number the job should move, where it sits today, and when you’ll check it. For an HR team at a company that doubled in size, it was days from offer accepted to fully onboarded, and it went from six to one. “Efficiency” with no number means nobody will know if it worked.

12. When would you tell us not to hire you?

Honest vendors have this one ready. Ours: a one-off project instead of weekly work, nobody who does the job can spare an hour a week, or you want a chatbot for your website. And if you have an engineer in-house with time to spare, building it yourself can be the right call.

Two weeks, give or take

The Money Pit ends fine. The house gets finished and it’s beautiful. It just took a lot longer, and a lot of yelling.

The next time somebody says two weeks, you’ll know whether they mean it.

PS, our answers to most of these are on the agents page, prices included. To run the whole list on us live, book an agent scoping call. It’s 30 minutes, and I’d rather you ask the hard ones.