Skip to content

Observability providers enable monitoring, tracing, and logging of agent operations. This helps you understand agent behavior, track performance, and debug issues in production.

Overview

AgentPool integrates with leading observability platforms:

  • Logfire: Pydantic's observability platform with native support for structured logging
  • Langsmith: LangChain's tracing and evaluation platform
  • AgentOps: Specialized platform for agent monitoring and analytics
  • Arize Phoenix: Open-source observability for LLM applications
  • Custom: Integrate your own observability solution

These integrations provide automatic instrumentation of agent operations, capturing traces, spans, and metrics without code changes.

Configuration Reference

Logfire

Configuration for Logfire endpoint.

Logfire (YAML)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
- type: logfire
  token: null  # Logfire token.
  region: us  # Logfire region.
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

Langsmith

Configuration for Langsmith endpoint.

Langsmith (YAML)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
- type: langsmith
  api_key: null  # Langsmith API key.
  project_name: null
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

AgentOps

Configuration for AgentOps endpoint.

AgentOps (YAML)
1
2
3
4
5
6
7
8
9
- type: agentops
  api_key: null  # AgentOps API key.
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

Arize Phoenix

Configuration for Arize Phoenix endpoint.

Arize Phoenix (YAML)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
- type: arize
  api_key: null  # Arize API key.
  space_key: null  # Arize space key.
  model_id: null  # Arize model ID.
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

Axiom

Configuration for Axiom endpoint.

Axiom (YAML)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
- type: axiom
  api_token: null  # Axiom API token with ingest permissions.
  dataset: traces  # Axiom dataset name where traces are sent.
  region: null  # Axiom region. If not set, uses default cloud endpoint.
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

Custom OTEL

Configuration for custom OTEL endpoint.

Custom OTEL (YAML)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
- type: custom
  endpoint: https://otel.example.com  # Custom OTEL endpoint URL.
  headers: {}  # Custom headers for the OTEL endpoint.
  enabled: true  # Provider enabled state.
  service_name: null
  environment: null
  protocol: http/protobuf
  instrument_pydantic_ai: false  # Enable logfire.instrument_pydantic_ai(). Disabled by default — manual spans cover critical paths.
  instrument_mcp: false  # Enable logfire.instrument_mcp(). Disabled by default — manual tool spans cover this.
  instrument_fastapi: true  # Enable logfire.instrument_fastapi() for HTTP server spans.

Key Features

  • Automatic tracing: All agent operations are automatically traced
  • Structured logging: Rich context and metadata for every operation
  • Performance metrics: Token usage, latency, and cost tracking
  • Error tracking: Detailed error capture with stack traces and context
  • Multi-provider support: Use multiple observability platforms simultaneously

Use Cases

  • Development: Debug agent behavior and optimize prompts
  • Production monitoring: Track performance and reliability metrics
  • Cost optimization: Analyze token usage and identify expensive operations
  • Quality assurance: Evaluate agent responses and identify issues
  • Compliance: Maintain audit logs of agent interactions

Configuration Notes

  • Providers can be configured at the manifest level for global observability
  • API keys should be stored in environment variables for security
  • Multiple providers can be active simultaneously
  • Each provider captures different metrics and views of agent behavior
  • Some providers offer additional features like prompt optimization and A/B testing