Telemetry¶
Lens can send traces and metrics about its own work to an OpenTelemetry collector, to see how it performs: which API requests are slow, how long each job step takes, how often jobs fail, and how many tokens model calls use and what they cost.
It is off unless you turn it on. Nothing is collected while it is off, and Lens has no built-in destination: it never sends telemetry to the Lens project or anyone else. Turned on, it sends only to the endpoint you set, typically a collector on the same machine or network, which can pass it on to Jaeger, Grafana Tempo and Prometheus, SigNoz, Honeycomb, or whatever you use.
To have Prometheus scrape Lens instead, see Prometheus.
Turning it on¶
Any of these:
- Settings → Telemetry: turn on Send telemetry and set the OTLP endpoint. Send a test span checks the address and headers first, even while it is off. The section shows whether it is on, where it sends, and how the last exports went.
- The setup wizard's Telemetry step, a click away on a fresh install (it isn't asked: off is its default).
- .env, for the API and every worker:
LENS_TELEMETRY=on # off keeps it off, whatever Settings say
LENS_TELEMETRY_ENDPOINT=http://localhost:4318 # the collector's OTLP/HTTP address
LENS_TELEMETRY_HEADERS=Authorization=Bearer%20xyz # optional: key=value pairs, comma-separated, URL-encoded
What .env sets wins over Settings, which show those fields locked.
Each process (the API, each lens worker, lens watch) follows the settings on its own and picks up a change within a
few seconds, with no restart. The endpoint is the collector's base address: traces go to /v1/traces and metrics to
/v1/metrics under it, over OTLP/HTTP with protobuf.
In Docker, localhost is the container itself: point the endpoint at a collector on the lens network (for example
http://otel-collector:4318, below) or at http://host.docker.internal:4318 for one on the host.
What is sent¶
| Signal | What | Attributes |
|---|---|---|
| Trace | each API request, named by its route template (GET /api/v1/recordings/{rid}) |
method, route, status code |
| Trace | each job, with a span per step (step transcribe) |
job, asset and pipeline ids; step type and outcome |
| Trace | routines and their actions, workflow runs | routine, workflow and asset ids; action type; trigger |
| Trace | each call to the model server (chat llama3.1) |
model, server host, temperature, tokens in and out, estimated cost |
| Metric | http.server.request.duration |
method, route, status code |
| Metric | lens.job.step.duration, lens.jobs |
step type, outcome |
| Metric | lens.routine.runs, lens.automation.runs, lens.workflow.runs |
outcome |
| Metric | gen_ai.client.operation.duration, gen_ai.client.token.usage |
model, token type |
| Metric | lens.llm.cost (USD) |
model |
Every span and metric carries service.name (Settings → Telemetry, default lens), service.version, a random
service.instance.id per process, and lens.process.role (api, worker or watcher). Model calls follow the
OpenTelemetry GenAI conventions.
Never sent: assets' text, prompts or answers, file names, titles, namespace or person names, people, email addresses, client IP addresses, request paths or query strings (only route templates), host names, or error messages (only the error's type, since a message can quote content).
Tokens and cost¶
Token counts come from the model server's reply (usage). Most servers report them; a streamed chat answer has them
only when the server sends usage in its last chunk.
Cost is an estimate from prices you set: Model prices in Settings → Telemetry, one line per model with its input
and output price in dollars per million tokens (gpt-4o-mini 0.15 0.60). The prices in Settings → AI assistant count
for the configured model too. A model without a price gets no cost, so local models cost nothing.
Other settings¶
| Setting | Default | |
|---|---|---|
| Traces / Metrics | on / on | send either or both |
| Share of traces kept | 1 |
0.1 keeps one trace in ten (head sampling) |
| Send metrics every | 60 seconds |
|
| Service name | lens |
how this server shows in your tracing tool |
In archive.yaml they live under telemetry: (enabled, endpoint, traces, metrics, sample_ratio,
export_seconds, service_name, prices).
A collector next to Lens¶
A minimal OpenTelemetry Collector that takes Lens's traces and metrics and
prints them (swap the debug exporter for your backend's):
# otel-collector.yaml
receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
exporters:
debug:
verbosity: basic
service:
pipelines:
traces: { receivers: [otlp], exporters: [debug] }
metrics: { receivers: [otlp], exporters: [debug] }
Beside docker-compose.prod.yml, in a docker-compose.override.yml or with -f:
services:
otel-collector:
image: otel/opentelemetry-collector:latest
command: ["--config=/etc/otel-collector.yaml"]
volumes:
- ./otel-collector.yaml:/etc/otel-collector.yaml:ro
networks: [lens]
Then set the endpoint to http://otel-collector:4318 and turn telemetry on.