Telemetry

Explore

Documentation

The orchestrator gathers and emits state information (telemetry) using an open-source project known as OpenTelementry (OTLP). You can read more about OTLP online.

To collect and review orchestrator telemetry, we recommend using a vendor’s OTLP collector (DataDog, NewRelic, etc.).

After configuring the orchestrator to emit metrics (below), you can setup a telemetry collector using our documentation.

Enabling telemetry

Each section section of the orchestator telemetry configuration is explained below. A complete example can be found at the end of this document.

telemetry

The telemetry key is the main config starting point for telemetry. When omitted, the orchestrator does not emit metrics.

metrics

The metrics key is required below telemetry. It contains all of the telemetry metrics configuration, described as keys below.

enabled

To emit metrics, add key enabled: true below metrics. To disable metrics output, simply change the enabled key value to false.

interval

Optionally configure the metrics reporting interval, add an interval key followed by a time duration: interval: 15m The default interval is 5m.

exporter

The required exporter key references an exporter name from the exporters section below. Orchestrator metrics will be emitted to the specified exporter value. The name specified here must match one defined in exporters configuration below. See the example at the end of this document.

health

The health key is required below telemetry.

enabled

To emit health, add key enabled: true below health. To disable health output, simply change the enabled key value to false.

exporter

The required exporter key references an exporter name from the exporters section below. Orchestrator health will be emitted to the specified exporter value. The name specified here must match one defined in exporters configuration below. See the example at the end of this document.

exporters

This required key describes where and how to export the Orchestrator metrics data. For more information about collectors where you can export telemetry data, see Collectors

The orchestrator currently outputs only to OTLP; the following section is required under exporters:

    otlp:
      protocols:

http

The orchestrator exports OTLP telemetry using an HTTP (only) exporter. Under http, add the endpoint key and specify where the orchestrator should emit metrics data:

        http:
          endpoint: localhost:4318

urlPath

By default, OTLP collectors listen to the URL path /v1/metrics for metrics data. However, OTLP collectors can also be configured instead to listen at different URL paths. When using an OTLP collector listening at a different or custom URL path, you must update the exporter configuration with the URL path expected by the collector.

To optionally specify a URL path for an exporter, use the urlPath key in the http section:

        http:
          endpoint: localhost:4318
          urlPath: /customPath

headers

Optionally add http headers which are emitted to the OTLP collector. Add headers key, specifying the header name and value to send with the HTTP request:

        http:
          headers:
            api-key: auth-header-value
            x-company-header: company-value

By default, no additional headers are output with metrics.

Complete example

Here is an example of a running orchestrator telemetry configuration:

tls:
  telem-tls:
    certFile: telemetry/localhost-client.pem
    keyFile: telemetry/localhost-client-key.pem
    caFile: telemetry/rootCA.pem

telemetry:
  metrics:
    enabled: true
    interval: 5m
    exporter: otlp/datadog
  health:
    enabled: true
    exporter: otlp/local
  exporters:
    otlp/local:
      protocols:
        http:
          endpoint: localhost:4318
          insecure: true
    otlp/datadog:
      protocols:
        http:
          tls: telem-tls 
          endpoint: otlp.nr-data.net:4318
          headers:
            api-key: <Your company's NewRelicAPIKey>