Tracing

​

Overview

Applying Opentelemetry Traces to the concept of workflows means observing the execution of the workflow and the individual tasks within it. Visualizing the traces produced by a workflow in a tool such as Axiom might look like the following:

By tracing your workflows you can easily observe:

• the order in which tasks are executed
• which tasks are executed in parallel
• which task runner is executing each task
• the duration of each task
• the outcome of each task (success, failure)

This can be useful to identify bottlenecks and performance issues, and to ensure that your workflows are executing as expected.

​

Configure tracing

The Tilebox workflow SDKs contain built-in support for exporting open telemetry traces. To enable tracing, simply call the corresponding configuration functions during the start up of your task runner.

• Axiom
• OpenTelemetry-compatible backend

from tilebox.workflows import Client
from tilebox.workflows.observability.tracing import configure_otel_tracing_axiom

# your own workflow:
from my_workflow import MyTask

def main():
    configure_otel_tracing_axiom(
        # specify an axiom dataset to export traces to
        dataset="my-axiom-traces-dataset",
        # along with an axiom api key with ingest permissions to that dataset
        api_key="my-axiom-api-key",
    )

    # below task runner will generate traces for the tasks it executes and
    # export trace and span data to the specified axiom dataset
    client = Client()
    runner = client.runner("dev-cluster", tasks=[MyTask])
    runner.run_forever()

if __name__ == "__main__":
    main()

As soon as the runner picks up tasks and executes them, corresponding traces and spans are automatically generated and exported to the configured backend.