Orchestrator health

The Orchestrator emits its health status using Open Telemetry. To learn more about Open Telemetry, see Telemetry configuration.

The Maverics platform automatically adds configuration to show orchestrator health in the UI when when it publishes configuration bundles. Health information can be sent to other Open Telemetry targets when configuring the orchestrator manually.

Health information

When configured, the Orchestrator emits a Health message (trace) every 5 minutes, named io.strata.maverics.health. See our complete example.

Each health message emitted by the Orchestrator includes the following trace attributes:

Attribute NameValue
start.timeStart time of the orchestrator (number of nanoseconds elapsed since January 1, 1970 UTC)
up.timeRuntime duration of the orchestrator, as elapsed from start_time (number of nanoseconds)
config.versionVersion number of the current orchestrator config file

Additionally, basic span properties are emitted via OTLP. These properties are:

Span PropertyValue
trace_idID of this trace
parent_idID of the parent of this trace- for orchestrator health this is currently empty.
idID of this span
start timeTime (UTC) when the trace started
end timeTimne (UTC) when the trace completed
status codeUnset
status message

Every message includes all of the default telemetry resource attributes. These resource attributes include detailed information about the Orchestrator. To learn more about the default resources, see Resources.

Complete example

 InstrumentationScope io.strata.maverics.health 
      -> start.time: Int(1688048617216828897)
      -> up.time: Int(45892)
 	{"kind": "exporter", "data_type": "traces", "name": "logging"}
 Span #0
     Trace ID       : 2656200024372ce0aced02f70766d18b
     Parent ID      : 
     ID             : ac4fd6896ca7e1a9
     Name           : orchestrator-health
     Kind           : Internal
     Start time     : 2023-06-29 14:23:37.216878661 +0000 UTC
     End time       : 2023-06-29 14:23:37.216888647 +0000 UTC
     Status code    : Unset
     Status message : 
 Resource SchemaURL: https://opentelemetry.io/schemas/1.17.0
 Resource attributes:
      -> container.id: Str(9b64fd552087)
      -> host.name: Str(localhost.localdomain)
      -> os.description: Str(Fedora Linux 35 (Workstation Edition) ...)
      -> os.type: Str(linux)
      -> process.command_args: Slice(["bin/maverics_linux_amd64","-v","-config","../maverics.yaml"])
      -> process.executable.name: Str(maverics_linux_amd64)
      -> process.executable.path: Str(/maverics/bin/maverics_linux_amd64)
      -> process.owner: Str(emo)
      -> process.pid: Int(1234567)
      -> process.runtime.description: Str(go version go1.19 linux/amd64)
      -> process.runtime.name: Str(go)
      -> process.runtime.version: Str(go1.19)
      -> service.instance.id: Str(11111111-2222-3333-4444-aaaaaaaaaaa)
      -> service.name: Str(io.strata.maverics)
      -> service.version: Str(0.22.36)