Logging
Set up distributed logging using the open telemetry logging protocol
Overview
Since Tilebox workflows are designed to be executed in a distributed fashion, properly setting up logging to a centralized logging system is a key aspect. Tilebox comes with built-in support for OpenTelemetry logging, which is an easy and convenient way to log messages from your tasks, and export them to a backend of your choice. Collecting and visualizing logs from a distributed cluster of task runners and visualizing them in a tool such as Axiom might look like the following:
Configure logging
The Tilebox workflow SDKs contain built-in support for exporting open telemetry logs. To enable logging, simply
call the corresponding configuration functions during the start up of your task runner
and then use to Tilebox provided logger
to emit log messages from your tasks.
To configure logging with Axiom, you first need to create a Axiom Dataset to export Tilebox workflow logs to. Additionally, an API key with ingest permissions to that dataset is required.
If you set environment variables AXIOM_API_KEY
and AXIOM_LOGS_DATASET
you can omit those arguments
to the configure_otel_logging_axiom
function.
Emitting log messages
The Tilebox SDKs provide a logger
that can be used to emit log messages from your tasks. Use it within
a tasks execute
method to emit log messages to the configured logging backend.
The logger
is a global singleton, so you can use it anywhere in your workflow. If you want to use a different
logger for a specific task, you can create a new logger using the get_logger
function.
Logging task runner internals
Tilebox Task runners also use such a logger internally. By default it’s configured to the WARNING level, but you can change this by explicitly configuring a logger for the workflows client constructing the task runner.
Was this page helpful?