Directory AI & ML PlatformsAI Governance & SafetyOpenLLMetry

OpenLLMetry

About OpenLLMetry

OpenLLMetry can be started by adding two lines of code.

From OpenLLMetry's own website, read 2026-10-24. How we build listings.

How to evaluate AI Governance & Safety

This is how the CIOPages Research Team evaluates this category. It is not an assessment of OpenLLMetry. It comes from our AI Governance & Responsible AI buyer guide.

25%

AI Inventory & Lifecycle Governance

Automated discovery and a registry of every model, LLM/prompt, agent, and embedded-AI feature (including shadow and third-party AI); risk tiering by use case; intake-to-retirement workflow with approvals, deployment gates, and named owners; versioning and change control

25%

Regulatory Mapping & Compliance Evidence

Out-of-the-box, maintained policy packs for the EU AI Act, NIST AI RMF, ISO 42001, SR 11-7, and sector rules (e.g. NYC LL144); control mapping and gap analysis; one underlying assessment that satisfies many frameworks; audit-ready, exportable evidence and model cards

20%

Model Monitoring & Explainability

Production monitoring for drift, performance decay, and data quality; bias and fairness testing across protected groups; explainability (e.g. SHAP / feature attribution) for tabular and NLP models; the depth of this domain is what separates observability tools from pure GRC

15%

GenAI & Agentic Oversight

LLM evaluation and guardrails (hallucination, toxicity, PII leakage, prompt-injection and jailbreak defense); agent discovery, runtime policy enforcement, and a logged trail of agent actions and the authority behind them; red-teaming and continuous evaluation of agent traces

10%

Integration & ML-Stack Fit

Connectors to your model platforms (SageMaker, Vertex AI, Databricks, Dataiku, MLflow, Bedrock), CI/CD and registries; API and policy-as-code coverage; identity (SSO/RBAC); fit with existing GRC and data-security tooling rather than yet another silo

Human-in-the-Loop & Accountability

Cross-functional workflows that reach risk, legal, and business owners (not just data scientists); reviewer sign-off and attestations; issue and exception tracking; reporting that a board or regulator can read, with a defensible audit trail

Other AI Governance & Safety Vendors

CIOPages put this listing together from public sources. It’s information, not an endorsement. How we build listings. Work here? Claim this listing or .