Skip to content
botshub

AI agent observability and evaluation tools

Agent observability and evaluation tools record, trace and test what LLM applications and agents do, so teams can debug failures, measure quality and monitor cost.

The leading AI agent observability and evaluation tools on botshub include ClawMetry, Opik and OrcaReplay. botshub lists 10 reviewed AI agent observability and evaluation tools from companies around the world.

Top 10 at a glance

  1. 1.
    ClawMetry

    See your agent think. Zero-config observability & governance for 30 AI agent runtimes: Claude Code, OpenAI Codex, Hermes, OpenClaw & 26 more. Live token costs, sessions, tool calls, crons. ...

  2. 2.
    Opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

  3. 3.
    OrcaReplay

    OrcaReplay β€” Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

  4. 4.
    LangFuse

    Trace, evaluate, and improve AI agents with one open platform. Use production data to understand behavior, collaborate on fixes, and ship better quality at lower cost and latency.

  5. 5.
    Arize Phoenix

    AI Observability & Evaluation

  6. 6.
    Kitaru

    Instill trust in your agent evals

  7. 7.
    AgentOps

    Every Agent Needs AgentOps.

  8. 8.
    Helicone

    AI Gateway & LLM Observability

  9. 9.
    LangSmith by LangChain

    LangSmith provides tools for debugging, testing, evaluating, and monitoring LLM applications, integrating seamlessly with LangChain for comprehensive AI agent observability

  10. 10.
    Pezzo

    πŸ•ΉοΈ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Compare the top 10

AI agent observability and evaluation tools: comparison of the top 10 listings
#NameMakerBased inTypeOpen source
1ClawMetryClawMetryNetherlands–Yes
2OpikOpik––Yes
3OrcaReplayOrcaReplayUnited States–Yes
4LangFuseLangFuseGermany–Yes
5Arize PhoenixArize Phoenix––Yes
6KitaruKitaruGermany–Yes
7AgentOpsAgentOpsUnited States–Yes
8HeliconeHeliconeUnited States–Yes
9LangSmithLangChainUnited States–Yes
10PezzoPezzoUnited States–Yes

Browse by country

ClawMetry

Netherlands

Agent observability and evaluation

See your agent think. Zero-config observability & governance for 30 AI agent runtimes: Claude Code, OpenAI Codex, Hermes, OpenClaw & 26 more. Live token costs, sessions, tool calls, crons. ...

Agent observability and evaluation

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

OrcaReplay

United States

Agent observability and evaluation

OrcaReplay β€” Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

LangFuse

Germany

Agent observability and evaluation

Trace, evaluate, and improve AI agents with one open platform. Use production data to understand behavior, collaborate on fixes, and ship better quality at lower cost and latency.

LangSmith

by LangChain

United States

Agent observability and evaluation

LangSmith provides tools for debugging, testing, evaluating, and monitoring LLM applications, integrating seamlessly with LangChain for comprehensive AI agent observability

Pezzo

United States

Agent observability and evaluation

πŸ•ΉοΈ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Frequently asked questions

What are the best AI agent observability and evaluation tools?
The leading AI agent observability and evaluation tools on botshub include ClawMetry, Opik and OrcaReplay. botshub lists 10 reviewed AI agent observability and evaluation tools from companies around the world.
What are AI agent observability and evaluation tools?
Agent observability and evaluation tools record, trace and test what LLM applications and agents do, so teams can debug failures, measure quality and monitor cost.
How many AI agent observability and evaluation tools are listed?
botshub currently lists 10 AI agent observability and evaluation tools. The most listings are from United States (5), Germany (2) and Netherlands (1).
Are there open-source AI agent observability and evaluation tools?
Yes. 10 of the 10 listings are tagged open source, including ClawMetry, Opik and OrcaReplay. Check each project's licence on its own page.
How are these listings ranked?
Listings are ordered by a quality score based on how complete and well documented each entry is. Every entry is reviewed before it is published.

Other categories