8 questions from the RFI stage, free
These come from the module as sold. The workbook adds follow-ups, a response format, a weight and a score column to each.
1. Confirm that customers can pin usage to a specific, immutable model version identifier. If so, describe your versioning scheme and the guarantee of immutability (i.e., that underlying weights and inference configuration will not change for a pinned version).
Why it matters. Stable, pinnable model versions are foundational for production reliability. Vendors may silently swap model weights behind a stable identifier, breaking customer workflows that were validated against a prior version. This question verifies the vendor's commitment to immutability, a critical control for managing behavioral drift and ensuring repeatable outcomes.
- Explicitly confirms customers can pin to an immutable version identifier
- Describes a clear versioning scheme (e.g., semantic, dated snapshots)
- Distinguishes between immutable (pinned) versions and floating aliases (e.g., 'latest')
- Only offers floating aliases with no option to pin
- Reserves the right to make silent updates behind a stable identifier
- Ambiguous definition of what constitutes a 'version'
2. Describe the categories and sources of data used to train or fine-tune the model(s) behind your product, including any use of public web data, licensed datasets, synthetic data, and customer data.
Why it matters. Training data provenance is the basis for downstream IP, copyright, privacy, and bias risk. Buyers need a high-level disclosure to triage exposure before deeper diligence. Vendors using third-party models should disclose what they can document about the upstream provider's data.
- Names data categories (e.g., web crawl, licensed corpora, synthetic, code, customer)
- Identifies major dataset sources where known or permissible
- Distinguishes pre-training, fine-tuning, and preference-tuning (RLHF/DPO) data
- Refuses to disclose training data sources even at a category level
- Cannot describe the training data composition for third-party models used
- Ambiguity about whether customer data is used for training
3. List the specific model(s) that power your product, including base model name, provider, version, and whether the model is vendor-developed, fine-tuned from a third-party base, or accessed via a third-party API.
Why it matters. Buyers cannot evaluate downstream risk (data flow, sub-processors, geopolitical exposure, IP indemnification) without knowing which models are in use. Vendors who obscure this disclosure transfer unmanaged risk to the customer.
- Provides a table with specific model names and versions
- Clearly identifies the underlying provider for each model
- Distinguishes own-trained, fine-tuned, and API-accessed models
- Refers to 'proprietary models' without disclosure
- Names a model family (e.g., 'GPT-4 class') without specific versions
- Omits sub-models used for embedding, classification, or moderation
4. Describe your internal model risk management framework, including whether it aligns to a recognized framework such as NIST AI RMF, ISO/IEC 42001, or an equivalent.
Why it matters. A documented model risk management (MRM) framework gives buyers a recognizable structure to map against their own governance. Alignment to recognized frameworks reduces audit burden and provides a common vocabulary.
- Names a specific framework (NIST AI RMF, ISO/IEC 42001, etc.) as their baseline
- Provides a mapping document or self-attestation against the framework's requirements (e.g., ISO/IEC 42001 Annex A controls; NIST AI RMF subcategories)
- Holds ISO/IEC 42001 certification from an accredited certification body, or has had an independent assessment against NIST AI RMF
- No formal framework is named
- Claims alignment without the ability to describe specifics or provide evidence
- No internal governance body or owner for model risk
5. What is your minimum advance-notice period for deprecating or sunsetting a model version that customers depend on?
Why it matters. Migrating an AI-dependent workload to a new model involves re-evaluation, regression testing and change management. Short deprecation windows force buyers into rushed migrations that bypass governance. The notice period is a concrete commitment the buyer can put in the contract.
- States a specific minimum notice period
- References a published deprecation policy
- Differentiates notice periods by customer tier or contract
- No defined notice period
- Reserves the right to deprecate at any time
- Notice period is shorter than the buyer's own migration and regression-testing cycle
6. Confirm whether customer prompts, inputs, outputs, or telemetry are used to train, fine-tune, or otherwise improve your models or any third-party model, and describe any opt-in/opt-out controls.
Why it matters. Use of customer data for training is one of the highest-impact governance questions: it determines whether confidential business data leaks into model weights. The default behavior, opt-in/opt-out semantics, and propagation to third-party providers all matter.
- A clear 'no' by default for enterprise customers
- Opt-in rather than opt-out for any training use
- Controls are guaranteed to propagate to all third-party model providers
- Default is to use customer data for training
- Opt-out is only available on a premium enterprise tier
- Cannot guarantee third-party providers honor the opt-out
7. Identify any third-party model providers that act as sub-processors for customer data passed to models, and where those providers process the data geographically.
Why it matters. Where the vendor processes personal data for the customer as a processor, a third-party model provider that receives that data is a sub-processor (GDPR Art. 28(2) and 28(4)). Their identity and processing location are needed for DPIA, data residency assessment, and cross-border transfer analysis.
- Names each model sub-processor (e.g., specific provider and service)
- Maps each sub-processor to the data categories it receives
- Identifies processing regions and data-transfer mechanisms
- No sub-processor disclosure for AI/model providers
- Bundles model providers into a generic 'cloud infrastructure' category
- Cannot confirm processing geography
8. Describe the pre-release evaluation regime applied to each new model version, including the categories of evaluations (capability, safety, bias, robustness) and whether results are shared with customers.
Why it matters. Pre-release evaluations are the primary control against shipping regressions or harms. The categories covered, methodology, and results shared enable downstream customer due diligence.
- Lists distinct evaluation categories (e.g., safety, bias, robustness, capability)
- Describes the methodology and benchmarks used for evaluation
- Includes red-teaming and adversarial evaluation as part of the process
- Evaluation is limited to capability benchmarks only
- No safety or bias evaluation is described
- No internal or external red-teaming process
What the audit changed
A language model drafted these questions and a second model critiqued them. Three audit passes followed and made 128 changes. Three examples:
Wrong or outdated citation
Draft: Third-party model providers are sub-processors under GDPR and equivalent regimes.
Now: Where the vendor processes personal data for the customer as a processor, a third-party model provider that receives that data is a sub-processor (GDPR Art. 28(2) and 28(4)).
A provider is a sub-processor only when the vendor is a processor and the provider processes personal data on its behalf. GDPR Art. 28(2) and 28(4) govern engaging 'another processor'.
Wrong or outdated citation
Draft: States a Service Level Agreement (SLA) for responding to requests
Now: States a response time that lets the customer meet GDPR Art. 12(3) (one month from receipt, extendable by two further months)
GDPR Art. 12(3) sets the controller's deadline for responding to a data-subject request.
Wrong or outdated citation
Draft: Provides a mapping document or self-attestation against the framework's controls
Now: Provides a mapping document or self-attestation against the framework's requirements (e.g., ISO/IEC 42001 Annex A controls; NIST AI RMF subcategories)
NIST AI RMF 1.0 is organized as functions, categories and subcategories, not controls; ISO/IEC 42001 Annex A lists controls.
