Directory AI & ML PlatformsAI Governance & SafetyOpenLIT

OpenLIT

About OpenLIT

OpenLIT provides an open-source agent harness engineering platform. It can be self-hosted free of charge under an Apache 2.0 license. The platform covers LLM tracing, agent evaluations, prompt management, and model comparison. It is built to be OpenTelemetry-native and work with models, frameworks, or harnesses. An Enterprise Edition adds additional capabilities for production teams. A fully hosted OpenLIT Cloud option is planned.

Products

LLM tracing: Provides OpenTelemetry LLM tracing for every call, tool, and retrieval, filterable by user, session, cost, latency, or metadata.
LLM evaluation: Performs LLM evaluation using LLM-as-a-judge, heuristics, or human review on production traffic or experiments.
Prompt Hub: Manages and versions prompts, allowing deployment and rollback without shipping app code.
OpenGround: Provides a model comparison playground to test prompts on real inputs and compare LLMs side by side.
Vault: Stores and rotates LLM API keys such as OpenAI and Anthropic secrets outside application code.
Fleet Hub: Monitors and configures OpenTelemetry collectors via OpAMP.

Used for

Tracing LLM calls, tool calls, and retrievals
Evaluating LLM outputs using LLM-as-a-judge, heuristics, or human review
Managing and deploying prompts with version control
Comparing LLMs side by side on prompts and real inputs
Monitoring GPU utilization, memory, temperature, and power for LLM inference

Who it is for

Used by developers and engineers.
Used by developers worldwide as part of an open, community-driven project.

From OpenLIT's own website, read 2026-10-24. How we build listings.

How to evaluate AI Governance & Safety

This is how the CIOPages Research Team evaluates this category. It is not an assessment of OpenLIT. It comes from our AI Governance & Responsible AI buyer guide.

25%

AI Inventory & Lifecycle Governance

Automated discovery and a registry of every model, LLM/prompt, agent, and embedded-AI feature (including shadow and third-party AI); risk tiering by use case; intake-to-retirement workflow with approvals, deployment gates, and named owners; versioning and change control

25%

Regulatory Mapping & Compliance Evidence

Out-of-the-box, maintained policy packs for the EU AI Act, NIST AI RMF, ISO 42001, SR 11-7, and sector rules (e.g. NYC LL144); control mapping and gap analysis; one underlying assessment that satisfies many frameworks; audit-ready, exportable evidence and model cards

20%

Model Monitoring & Explainability

Production monitoring for drift, performance decay, and data quality; bias and fairness testing across protected groups; explainability (e.g. SHAP / feature attribution) for tabular and NLP models; the depth of this domain is what separates observability tools from pure GRC

15%

GenAI & Agentic Oversight

LLM evaluation and guardrails (hallucination, toxicity, PII leakage, prompt-injection and jailbreak defense); agent discovery, runtime policy enforcement, and a logged trail of agent actions and the authority behind them; red-teaming and continuous evaluation of agent traces

10%

Integration & ML-Stack Fit

Connectors to your model platforms (SageMaker, Vertex AI, Databricks, Dataiku, MLflow, Bedrock), CI/CD and registries; API and policy-as-code coverage; identity (SSO/RBAC); fit with existing GRC and data-security tooling rather than yet another silo

Human-in-the-Loop & Accountability

Cross-functional workflows that reach risk, legal, and business owners (not just data scientists); reviewer sign-off and attestations; issue and exception tracking; reporting that a board or regulator can read, with a defensible audit trail

Other AI Governance & Safety Vendors

CIOPages put this listing together from public sources. It’s information, not an endorsement. How we build listings. Work here? Claim this listing or .