> ## Documentation Index
> Fetch the complete documentation index at: https://docs.strata.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Traces

The Orchestrator supports OpenTelemetry distributed tracing, exporting trace data via the OTLP protocol. Distributed tracing is in early rollout -- initial instrumentation covers specific subsystems, with coverage expanding in future releases.

<Info>
  Distributed tracing is currently instrumented for **Clustering** and **LDAP Provider** operations. Trace coverage will expand to additional Orchestrator subsystems in future releases.
</Info>

## Current Trace Coverage

The following Orchestrator subsystems currently emit distributed trace spans:

* **Clustering** -- Cluster coordination operations such as leader election, node membership, and state synchronization produce spans that help diagnose multi-instance behavior
* **LDAP Provider** -- LDAP bind, search, and connection operations produce spans that capture timing and status for upstream directory interactions

## Planned Coverage

Distributed tracing will expand to cover additional Orchestrator subsystems in future releases, including authentication flows, route matching, connector operations, authorization decisions, and response handling. Future releases also plan to propagate trace context headers on external requests, enabling end-to-end cross-service visibility.

## Setup

<Note>
  If you manage Orchestrator configuration through the Maverics Console, advanced telemetry settings require the [**config override**](/reference/console/config-publishing#override-config) feature. Config override requires enablement for your organization -- contact your Strata account team or [Strata support](https://strataidentity.my.site.com/support/s/) to enable it.
</Note>

Configure processors to export traces to your OTLP endpoint. The simple processor exports each span immediately; for production batch processing, see [Batch Processor](#batch-processor) below.

```yaml maverics.yaml theme={null}
telemetry:
  traces:
    processors:
      - simple:
          exporter:
            otlp:
              protocol: "http/protobuf"
              endpoint: "http://otelcol.example.com:4318/v1/traces"
```

| Field      | Description                                      |
| ---------- | ------------------------------------------------ |
| `protocol` | OTLP transport protocol -- use `"http/protobuf"` |
| `endpoint` | Your OTLP collector's traces endpoint URL        |

## OTLP Export

All traces are exported via OTLP (`http/protobuf`), the OpenTelemetry standard protocol. Any OTLP-compatible backend can receive trace data:

* **Jaeger** -- Popular open-source distributed tracing platform with native OTLP support
* **Grafana Tempo** -- Scalable trace backend for the Grafana stack
* **Datadog APM** -- Enable the OTLP receiver on the Datadog Agent for trace ingestion
* **New Relic** -- Send OTLP traces to New Relic's OTLP endpoint
* **Honeycomb** -- Native OTLP ingestion for traces and metrics
* **Zipkin** -- Use an OpenTelemetry Collector to translate OTLP traces to Zipkin format

<Tip>
  Deploy an OpenTelemetry Collector between the Orchestrator and your tracing backend. The Collector provides buffering, retry logic, and the ability to fan out traces to multiple backends simultaneously.
</Tip>

## Batch Processor

For production environments with high trace volume, the batch processor groups spans and exports them in batches, reducing network overhead and providing backpressure handling.

```yaml maverics.yaml theme={null}
telemetry:
  traces:
    processors:
      - batch:
          exporter:
            otlp:
              protocol: "http/protobuf"
              endpoint: "http://otelcol.example.com:4318/v1/traces"
              compression: "gzip"
          schedule_delay: 5000
          export_timeout: 30000
          max_queue_size: 2048
          max_export_batch_size: 512
```

| Field                   | Description                                    |
| ----------------------- | ---------------------------------------------- |
| `schedule_delay`        | Delay between batch exports in milliseconds    |
| `export_timeout`        | Timeout for each batch export in milliseconds  |
| `max_queue_size`        | Maximum number of spans queued before dropping |
| `max_export_batch_size` | Maximum number of spans per export batch       |

The simple processor is recommended for development and low-volume environments. The batch processor is recommended for production deployments with significant trace volume.

## Configuration Reference

### Simple Processor

The simple processor exports each span immediately as it completes. This is recommended for development and low-volume environments.

| Key                                                                     | Type    | Default | Required | Description                                                                                  |
| ----------------------------------------------------------------------- | ------- | ------- | -------- | -------------------------------------------------------------------------------------------- |
| `telemetry.traces.processors[].simple.exporter.otlp.protocol`           | string  | --      | Yes      | OTLP protocol -- `"http/protobuf"`                                                           |
| `telemetry.traces.processors[].simple.exporter.otlp.endpoint`           | string  | --      | Yes      | OTLP collector endpoint URL                                                                  |
| `telemetry.traces.processors[].simple.exporter.otlp.insecure`           | boolean | `false` | No       | Skip TLS verification when connecting to the collector                                       |
| `telemetry.traces.processors[].simple.exporter.otlp.timeout`            | integer | --      | No       | Export timeout in milliseconds                                                               |
| `telemetry.traces.processors[].simple.exporter.otlp.compression`        | string  | --      | No       | Compression type -- `"gzip"` or `"none"`                                                     |
| `telemetry.traces.processors[].simple.exporter.otlp.headers[]`          | array   | `[]`    | No       | Custom HTTP headers sent with each export request. Each entry has `name` and `value` fields. |
| `telemetry.traces.processors[].simple.exporter.otlp.certificate`        | string  | --      | No       | Path to CA certificate for TLS verification of the collector                                 |
| `telemetry.traces.processors[].simple.exporter.otlp.client_certificate` | string  | --      | No       | Path to client certificate for mTLS authentication                                           |
| `telemetry.traces.processors[].simple.exporter.otlp.client_key`         | string  | --      | No       | Path to client private key for mTLS authentication                                           |

### Batch Processor

The batch processor groups spans and exports them in batches. This is recommended for production environments with significant trace volume, as it reduces network overhead and provides backpressure handling.

| Key                                                                    | Type    | Default | Required | Description                                                                                  |
| ---------------------------------------------------------------------- | ------- | ------- | -------- | -------------------------------------------------------------------------------------------- |
| `telemetry.traces.processors[].batch.exporter.otlp.protocol`           | string  | --      | Yes      | OTLP protocol -- `"http/protobuf"`                                                           |
| `telemetry.traces.processors[].batch.exporter.otlp.endpoint`           | string  | --      | Yes      | OTLP collector endpoint URL                                                                  |
| `telemetry.traces.processors[].batch.exporter.otlp.insecure`           | boolean | `false` | No       | Skip TLS verification when connecting to the collector                                       |
| `telemetry.traces.processors[].batch.exporter.otlp.timeout`            | integer | --      | No       | Export timeout in milliseconds                                                               |
| `telemetry.traces.processors[].batch.exporter.otlp.compression`        | string  | --      | No       | Compression type -- `"gzip"` or `"none"`                                                     |
| `telemetry.traces.processors[].batch.exporter.otlp.headers[]`          | array   | `[]`    | No       | Custom HTTP headers sent with each export request. Each entry has `name` and `value` fields. |
| `telemetry.traces.processors[].batch.exporter.otlp.certificate`        | string  | --      | No       | Path to CA certificate for TLS verification of the collector                                 |
| `telemetry.traces.processors[].batch.exporter.otlp.client_certificate` | string  | --      | No       | Path to client certificate for mTLS authentication                                           |
| `telemetry.traces.processors[].batch.exporter.otlp.client_key`         | string  | --      | No       | Path to client private key for mTLS authentication                                           |
| `telemetry.traces.processors[].batch.schedule_delay`                   | integer | --      | No       | Delay between batch exports in milliseconds                                                  |
| `telemetry.traces.processors[].batch.export_timeout`                   | integer | --      | No       | Timeout for each batch export in milliseconds                                                |
| `telemetry.traces.processors[].batch.max_queue_size`                   | integer | --      | No       | Maximum number of spans queued before dropping                                               |
| `telemetry.traces.processors[].batch.max_export_batch_size`            | integer | --      | No       | Maximum number of spans per export batch                                                     |

<Note>
  Both `simple` and `batch` exporters also support `console` and `zipkin` exporter types in the SDK spec, but `otlp` is the recommended approach for production use.
</Note>

### Example

<Tabs>
  <Tab title="Console UI">
    <Info>
      **Console UI documentation is coming soon.** This section will walk you
      through configuring this component using the Maverics Console's visual
      interface, including step-by-step screenshots and field descriptions.
    </Info>
  </Tab>

  <Tab title="Configuration">
    Simple processor for development:

    ```yaml maverics.yaml theme={null}
    telemetry:
      traces:
        processors:
          - simple:
              exporter:
                otlp:
                  protocol: "http/protobuf"
                  endpoint: "http://otelcol:4318/v1/traces"
    ```
  </Tab>
</Tabs>

#### Advanced Example

<Tabs>
  <Tab title="Console UI">
    <Info>
      **Console UI documentation is coming soon.** This section will walk you
      through configuring this component using the Maverics Console's visual
      interface, including step-by-step screenshots and field descriptions.
    </Info>
  </Tab>

  <Tab title="Configuration">
    Batch processor with gzip compression, custom headers, and TLS:

    ```yaml maverics.yaml theme={null}
    telemetry:
      traces:
        processors:
          - batch:
              exporter:
                otlp:
                  protocol: "http/protobuf"
                  endpoint: "http://otelcol:4318/v1/traces"
                  compression: "gzip"
                  headers:
                    - name: "x-api-key"
                      value: "your-api-key"
              schedule_delay: 5000
              export_timeout: 30000
              max_queue_size: 2048
              max_export_batch_size: 512
    ```
  </Tab>
</Tabs>

## Related Pages

<CardGroup cols={2}>
  <Card title="Monitor and Observe" icon="eye" href="/guides/operations/monitor">
    Step-by-step guide for setting up telemetry in production
  </Card>

  <Card title="Metrics" icon="chart-line" href="/reference/orchestrator/telemetry/metrics">
    OpenTelemetry metrics via OTLP
  </Card>

  <Card title="Logging" icon="file-lines" href="/reference/orchestrator/telemetry/logging">
    Structured operational and auth event logging
  </Card>

  <Card title="Orchestrator Configuration" icon="sliders" href="/reference/orchestrator/configuration">
    Config file structure, environment variables, and runtime settings
  </Card>
</CardGroup>