Tracing

​

Overview

Applying OpenTelemetry traces to the concept of workflows allows you to monitor the execution of your jobs and their individual tasks. Visualizing the trace for a job in a tool like Axiom may look like this:

Tracing your workflows enables you to easily observe:

• The order of task execution
• Which tasks run in parallel
• The task runner handling each task
• The duration of each task
• The outcome of each task (success or failure)

This information helps identify bottlenecks and performance issues, ensuring that your workflows execute correctly.

​

Configure tracing

The Tilebox workflow SDKs have built-in support for exporting OpenTelemetry traces. To enable tracing, call the appropriate configuration functions during the startup of your task runner.

from tilebox.workflows import Client
from tilebox.workflows.observability.tracing import configure_otel_tracing_axiom

# your own workflow:
from my_workflow import MyTask

def main():
    configure_otel_tracing_axiom(
        # specify an Axiom dataset to export traces to
        dataset="my-axiom-traces-dataset",
        # along with an Axiom API key for ingest permissions
        api_key="my-axiom-api-key",
    )

    # the following task runner generates traces for executed tasks and
    # exports trace and span data to the specified Axiom dataset
    client = Client()
    runner = client.runner("dev-cluster", tasks=[MyTask])
    runner.run_forever()

if __name__ == "__main__":
    main()

Once the runner picks up tasks and executes them, corresponding traces and spans are automatically generated and exported to the configured backend.