The idea behind it
A standard security questionnaire asks about encryption, access control and audits. An AI product raises questions it does not cover: whether your prompts and files train the vendor's model or a third party's, which model provider sits behind the product, how long prompts and outputs are kept, how much notice you get before the model changes, who owns the output, and what role the vendor plays under the EU AI Act. A vendor can answer every security question well and still leave these open.
The questionnaire asks them in ten groups and gives the evaluator what to look for in each answer: why the question matters, what a good answer looks like and a red flag. Scoring separates a commitment in the contract from one on a policy page the vendor can change. Some answers rest on law: GDPR Article 28(2) requires a processor to have the controller's authorization before engaging another processor, and Article 33 puts a 72-hour clock on the controller's breach notice, which starts from the vendor's notice to you.
What's inside
- Ten question groups: data use, subprocessors and model providers, security and certifications, model changes and notice, evaluation evidence, output ownership and IP indemnity, incident notice, pricing and usage limits, exit and data return, and regulatory position under the EU AI Act.
- For each of the 38 questions: why it matters, what a good answer looks like and a red flag.
- A 0-to-3 scoring scale that separates a good answer with evidence from one stated only on a policy page.
- Must-have questions that flag a vendor scoring 0 on them, whatever its total.
- A worked example scoring three fictional vendors of a customer-service assistant, with the answers that decided it.
- The Word version splits into a questionnaire to send to vendors and an internal evaluator's guide.
- The Excel version scores up to four vendors and weights the group averages into a score out of 100.
Format: PDF (7 pages), Excel workbook and Word document.
A look inside
Pages 1 and 2 of 7 of the PDF. Select an image to enlarge it.
Who uses it
- Procurement and vendor managers, to send every shortlisted AI vendor the same questions and compare the answers on one scale.
- Security and privacy leads, to check data use, subprocessors, residency and incident notice before a contract is signed.
- Legal counsel, to turn the answers into contract terms on training, deletion, indemnity and exit.
- Business owners of the AI use case, to agree the weights and the must-have questions before the answers come back.
When to use it
- An AI product or an AI feature in an existing product is on a shortlist.
- A renewal adds AI features to a product you already license.
- A vendor changes the model provider behind its product or its data-use terms.
- A third-party risk review needs AI-specific questions beside the standard security questionnaire.
How to use it
Send the Word questionnaire to each shortlisted vendor, score the answers in Excel, and put the answers you relied on into the contract.
- Agree the group weights and the must-have questions with security, privacy, legal and the business owner before sending anything.
- Send Part 1 of the Word questionnaire to each vendor and ask for evidence with each answer: the clause, the report or the document.
- Score each answer 0 to 3 against the good answer and the red flag. Have two people score separately and settle the differences.
- Enter the scores in the Excel workbook and read the summary: the weighted score, the must-have failures and the questions still unscored.
- Go back to vendors on the answers that scored 1, and put every answer you relied on into the contract.
You leave with: Each vendor's answers to the same 38 questions with the evidence, a weighted score out of 100, the must-have failures, and a list of the terms to write into the contract.
Where it goes wrong
- Accepting a policy page as the answer. A no-training commitment the vendor can change without notice is not a contract term.
- Setting the weights after reading the answers, which turns the score into a justification for a vendor already chosen.
- Asking only about the vendor and not the model provider behind it, whose terms on training and retention may differ.
- Letting a high total hide a must-have failure, such as a vendor that will not name its model provider.
Get the questionnaire and scorecard
It is free. Enter your work email and we send the download link.
Questions
What should you ask an AI vendor about your data?
Whether your prompts, outputs and files train any model, under the contract and for every product tier you will use; how long each type of data is kept; how it is deleted, including embeddings and fine-tuned models; where it is stored and processed, inference included; and which of the vendor's staff can see it.
Why ask an AI vendor to name its model provider?
The model provider usually processes your data, so under GDPR Article 28(2) it is a subprocessor the vendor needs your authorization to use. Its terms on training, retention and residency also decide whether the vendor's commitments to you hold.
How is the weighted score calculated?
Each answer scores 0 to 3. Each group's score is the average of its scored questions. The weighted score is the sum of each group's weight times its average, divided by 3, so a vendor scoring 3 on every question scores 100. A vendor that scores 0 on a must-have question is flagged whatever its total.