Tracing, monitoring and evals for LLM apps and agents.
Open source agent evals and observability: trace, evaluate and improve LLM applications.
Pricing: Cloud: Hobby free (50k units/mo, 30 days data, 2 users); Core $29/mo; Pro $199/mo; Enterprise $2,499/mo. Extra usage from $8 per 100k units. Self-hosting is free. Source: langfuse.com/pricing
Engineering note: Billing counts traces, observations and scores as units, so a chatty multi-step agent consumes units fast. Supports Python/JS SDKs and OpenTelemetry. Includes prompt management and datasets.
AI observability and evaluation.
Pricing: not verified yet, check the tool's site
Engineering note: Repo lives under the Arize-ai org. Verify the self-host and hosted options in their docs.
Open source LLM observability platform. One line of code to monitor, evaluate and experiment.
Pricing: Hobby: 10,000 free requests, 7-day retention, 1 seat. Pro $79/mo; Team $799/mo; both usage-based on top. Source: helicone.ai/pricing
Engineering note: Integration is described as one line of code. Pricing page also lists gateway features such as caching, rate limits and fallbacks.
Open-source observability for your GenAI or LLM application, based on OpenTelemetry.
Pricing: not verified yet, check the tool's site
Engineering note: Built on OpenTelemetry, which helps if you already ship traces to an existing backend.
Debug, evaluate and monitor LLM applications, RAG systems and agentic workflows with tracing, automated evaluations and production dashboards.
Pricing: not verified yet, check the tool's site
Engineering note: Repo lives under the comet-ml org. Combines tracing, evals and dashboards in one tool.
LangChain's platform for tracing, evaluation, deployment and agent tooling.
Pricing: Developer $0/seat (5k base traces/mo); Plus $39/seat/mo (10k base traces/mo); Enterprise custom. Usage in LSUs at $1.00 each. Source: langchain.com/pricing
Engineering note: Base traces are retained 14 days; extended traces 180 days for an extra fee. Pricing page says LangSmith does not train on your data.