AI Vendor Evaluation Checklist

A standard vendor questionnaire asks about certifications and uptime. Those matter, and they are not where AI purchases usually go wrong.

This checklist is written from the buyer’s side. It covers the risks I write about: pricing that rests on subsidized compute, metering you cannot audit, a demo that measures the vendor’s expert, a supplier who may ship your product, and what happens to your secrets when the vendor is breached.

What is inside

• How to use it: send it before the demo, and what a refusal to answer tells you
• The questions, grouped by fit and proof, pricing, data and security, operations, agents, lock-in and the vendor’s own business
• For each question, what a good answer sounds like
• A scoring sheet to compare up to three vendors
• Red flags
• What to put in the contract

Get all 13 pages, free

Common questions

What questions should you ask an AI vendor?

Ask how the result was proven on work like yours, what the bill looks like at your production volume and whether you can audit it, what happens to your data and secrets if the vendor is breached, how you would switch away, and whether the vendor could ship a product that competes with yours.

How do you evaluate AI vendors?

Send the same questions to every vendor before the demo, score the written answers, and run a pilot on your own work with your own people. A demo run by the vendor’s expert mostly measures the expert.

Does this replace a security questionnaire?

No. Use it alongside your security review. It covers the commercial and operational risks that security questionnaires usually miss.