Tracing

Tilebox traces workflow jobs automatically. Job submission creates a root trace, runners continue that trace across machines, and every task execution creates a span.

Built-in traces connect task order, dependencies, parallel execution, task duration, task status, runner identity, service identity, and logs emitted while a span was active.

Add custom spans

Use context.tracer inside a task to add spans around meaningful parts of your own code.

from tilebox.workflows import ExecutionContext, Task

class ProcessScene(Task):
    scene_id: str

    def execute(self, context: ExecutionContext) -> None:
        with context.tracer.span("download-scene") as span:
            span.set_attribute("scene_id", self.scene_id)
            # download input data

        with context.tracer.span("compute-index"):
            # perform expensive computation
            pass

package tasks

import (
	"context"

	"github.com/tilebox/tilebox-go/workflows/v1"
)

type ProcessScene struct{}

func (t *ProcessScene) Execute(ctx context.Context) error {
	return workflows.WithSpan(ctx, "compute-index", func(ctx context.Context) error {
		// perform expensive computation
		return nil
	})
}

Custom spans are nested under the current task span. Logs emitted inside the span are correlated with its trace_id and span_id.

Span status and exceptions

If a task raises an exception, Tilebox records the exception on the task span and marks the span as failed before the task is retried or marked failed. For finer-grained error reporting, record errors on your custom spans before re-raising them.

class ProcessScene(Task):
    scene_id: str

    def execute(self, context: ExecutionContext) -> None:
        with context.tracer.span("publish-output") as span:
            try:
                # publish output
                pass
            except Exception as error:
                span.record_exception(error)
                raise

package tasks

import (
	"context"
	"fmt"

	"github.com/tilebox/tilebox-go/workflows/v1"
)

type ProcessScene struct{}

func (t *ProcessScene) Execute(ctx context.Context) error {
	return workflows.WithSpan(ctx, "publish-output", func(ctx context.Context) error {
		if err := publishOutput(); err != nil {
			return fmt.Errorf("failed to publish output: %w", err)
		}
		return nil
	})
}

func publishOutput() error {
	return nil
}

Query spans

You can retrieve spans for a job through the jobs client. Python results can also be converted to a pandas DataFrame.

from tilebox.workflows import Client

client = Client()
job = client.jobs().submit("process-scene", ProcessScene(scene_id="S2A_001"))

spans = client.jobs().query_spans(job)
for span in spans:
    print(span.name, span.status_code, span.duration)

df = spans.to_pandas()

package main

import (
	"context"
	"fmt"
	"log/slog"
	"time"

	"github.com/google/uuid"
	"github.com/tilebox/tilebox-go/workflows/v1"
)

func main() {
	ctx := context.Background()
	client := workflows.NewClient()
	jobID := uuid.MustParse("019e07b1-916b-0630-f3ba-f1c33235d174")

	for span, err := range client.Jobs.QuerySpans(ctx, jobID) {
		if err != nil {
			slog.ErrorContext(ctx, "failed to query job spans", slog.Any("error", err))
			return
		}

		fmt.Printf("%s %-40s %s\n",
			span.StartTime.Format(time.RFC3339Nano),
			span.Name,
			span.Duration(),
		)
	}
}

See Query telemetry for the log and span query APIs.

Export to another backend

Tilebox stores traces by default. To export spans to your own observability backend as well, configure an OpenTelemetry or Axiom integration when the runner process starts.

Get Started

Agents and AI Tools

Datasets

Workflows

Add custom spans

Span status and exceptions

Query spans

Export to another backend

​Add custom spans

​Span status and exceptions

​Query spans

​Export to another backend

Add custom spans

Span status and exceptions

Query spans

Export to another backend