1. Describe your methodology for evaluating bias in the model(s) powering your product. Include the benchmarks and datasets used, the cadence of re-evaluation, and triggers for out-of-cycle reviews.
Why it matters. Bias claims without documented methodology cannot be validated or compared across vendors. Buyers need to know whether the vendor treats bias evaluation as a continuous process tied to model changes, not a one-off exercise.
- Names specific public or internal benchmarks used (e.g. BBQ, BOLD, CrowS-Pairs, WinoBias).
- Describes statistical tests and significance thresholds used.
- Defines a regular cadence for re-evaluation (e.g., per model release, quarterly).
- Generic claims of 'rigorous testing' with no named benchmarks or metrics.
- Treats bias evaluation as a one-time pre-launch activity with no defined re-evaluation cadence.
- Cadence is described as 'as needed' with no objective triggers.