Datasets

You can create your own, Custom Datasets via the Tilebox Console.

Build a spatio-temporal catalog

Learn how to create a custom dataset catalog with the Python SDK.

Ingest into a spatio-temporal catalog

Learn how to ingest GeoParquet metadata into an existing spatio-temporal catalog.

Dataset types

Each dataset is of a specific type. Each dataset type comes with a set of required fields for each data point. The dataset type also determines the query capabilities for a dataset, for example, whether a dataset supports time-based queries or additionally also spatially filtered queries. To find out which fields are required for each dataset type check out the documentation for the available dataset types below.

Timeseries Data

Each data point is linked to a specific point in time. Common for satellite telemetry, or other time-based data. Supports efficient time-based queries.

Spatio-temporal Data

Each data point is linked to a specific point in time and a location on the Earth’s surface. Common for satellite imagery. Supports efficient time-based and spatially filtered queries.

Dataset specific fields

Additionally, each dataset has a set of fields that are specific to that dataset. Fields are defined during dataset creation. That way, all data points in a dataset are strongly typed and are validated during ingestion. The required fields of the dataset type, as well as the custom fields specific to each dataset together make up the dataset schema. Once a dataset schema is defined, existing fields cannot be removed or edited as soon as data has been ingested into it. You can always add new fields to a dataset, since all fields are always optional.

The only exception to this rule are empty datasets. If you empty all collections in a dataset, you can freely edit the data schema, since no conflicts with existing data points can occur.

Field types

When defining the data schema, you can specify each field’s type. The following field types are supported.

Primitives

Type	Description	Example value
string	A string of characters of arbitrary length.	`Some string`
int64	A 64-bit signed integer.	`123`
uint64	A 64-bit unsigned integer.	`123`
float64	A 64-bit floating-point number.	`123.45`
bool	A boolean.	`true`
bytes	A sequence of arbitrary length bytes.	`0xAF1E28D4`

Time

Type	Description	Example value
Duration	A signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. See Duration for more information.	`12s 345ms`
Timestamp	A point in time, represented as seconds and fractions of seconds at nanosecond resolution in UTC Epoch time. See Timestamp for more information.	`2023-05-17T14:30:00Z`

Identifier

Type	Description	Example value
UUID	A universally unique identifier (UUID).	`126a2531-c98d-4e06-815a-34bc5b1228cc`

Geospatial

Type	Description	Example value
Geometry	Geospatial geometries of type Point, LineString, Polygon or MultiPolygon.	`POLYGON ((12.3 -5.4, 12.5 -5.4, ...))`

Arrays

Every type is also available as an array, allowing to ingest multiple values of the underlying type for each data point. The size of the array is flexible, and can be different for each data point.

Listing datasets

You can use your client instance to access the datasets available to you. To list all available datasets, use the datasets method of the client.

from tilebox.datasets import Client

client = Client()
datasets = client.datasets()
print(datasets)

package main

import (
  "context"
  "log"

  "github.com/tilebox/tilebox-go/datasets/v1"
)

func main() {
  ctx := context.Background()

  client := datasets.NewClient()

  allDatasets, err := client.Datasets.List(ctx)
  if err != nil {
    log.Fatalf("Failed to list datasets: %v", err)
  }

  for _, dataset := range allDatasets {
    log.Println(dataset)
  }
}

Output

open_data:
    asf:
        ers_sar: European Remote Sensing Satellite (ERS) Synthetic Aperture Radar (SAR) Granules
    copernicus:
        landsat8_oli_tirs: Landsat-8 is part of the long-running Landsat programme ...
        sentinel1_sar: The Sentinel-1 mission is the European Radar Observatory for the ...
        sentinel2_msi: Sentinel-2 is equipped with an optical instrument payload that samples ...
        sentinel3_olci: OLCI (Ocean and Land Colour Instrument) is an optical instrument used to ...
        ...

Once you have your dataset object, you can use it to list the available collections for the dataset.

In python, if you’re using an IDE or an interactive environment with auto-complete, you can use it on your client instance to discover the datasets available to you. Type client. and trigger auto-complete after the dot to do so.

Creating / Updating a dataset

You can create a dataset using one of the available client SDKs, or use the Tilebox Console when you prefer a visual schema editor.

from datetime import timedelta
from tilebox.datasets import Client
from tilebox.datasets.data.datasets import DatasetKind

client = Client()

dataset = client.create_or_update_dataset(
    DatasetKind.SPATIOTEMPORAL,
    "my_landsat8_oli_tirs",
    [
        {
            "name": "granule_name",
            "type": str,
            "description": "The name of the Landsat-8 granule.",
            "example_value": "LE07_L1TP_174061_20010913_20200917_02_T1",
        },
        {
            "name": "cloud_cover",
            "type": float,
            "description": "Cloud cover percentage",
            "example_value": "91",
        },
        {
            "name": "proj_shape",
            "type": list[int],
            "description": "Raster shape",
            "example_value": "[6971, 7801]",
        },
    ],
    name="Personal Landsat-8 catalog",
)

dataset, err := client.Datasets.Create(ctx,
  datasets.KindSpatiotemporal,
  "my_landsat8_oli_tirs",
  "Personal Landsat-8 catalog",
  []datasets.Field{
    field.String("granule_name").Description("The name of the Earth View Granule").ExampleValue("20220830_185202_SN18_10N_552149_5297327"),
    field.Float64("cloud_cover").Description("Cloud cover percentage").ExampleValue("91"),
    field.Int64("proj_shape").Repeated().Description("Raster shape").ExampleValue("[6971, 7801]"),
  }
)

This code will create a new catalog with 7 fields in total. 4 of those fields are auto-generated by choosing the spatio-temporal dataset type, and 3 (granule_name, cloud_cover, proj_shape) are custom fields that are defined.

If a dataset with the same code_name already exists, it will be updated instead.

Accessing a dataset

Each dataset has an automatically generated slug that can be used to access it. The slug is the name of the group, followed by a dot, followed by the dataset code name. For example, the slug for the Sentinel-2 MSI dataset, which is part of the open_data.copernicus group, is open_data.copernicus.sentinel2_msi. To access a dataset, use the dataset method of your client instance and pass the slug of the dataset as an argument.

from tilebox.datasets import Client

client = Client()
s2_msi_dataset = client.dataset("open_data.copernicus.sentinel2_msi")

s2MsiDataset, err := client.Datasets.Get(ctx, "open_data.copernicus.sentinel2_msi")

Once you have your dataset object, you can use it to access available collections for the dataset.

Deleting a dataset

Datasets can be deleted through the Tilebox Console by clicking the Delete button in the dataset page. Empty datasets will be deleted right away. A dataset is considered empty if none of its collections contain any data points. A non-empty dataset can also be deleted, but is a destructive operation. Every data point in every collection of the dataset will be deleted. As a safety measure, Tilebox soft-deletes the dataset for 7 days before permanently deleting it. Please get in touch if you deleted a dataset by accident and want to restore it.

Get Started

Agents and AI Tools

Workflows

Datasets

Build a spatio-temporal catalog

Ingest into a spatio-temporal catalog

Dataset types

Timeseries Data

Spatio-temporal Data

Dataset specific fields

Field types

Primitives

Time

Identifier

Geospatial

Arrays

Listing datasets

Creating / Updating a dataset

Accessing a dataset

Deleting a dataset

​Related Guides

Build a spatio-temporal catalog

Ingest into a spatio-temporal catalog

​Dataset types

Timeseries Data

Spatio-temporal Data

​Dataset specific fields

​Field types

​Primitives

​Time

​Identifier

​Geospatial

​Arrays

​Listing datasets

​Creating / Updating a dataset

​Accessing a dataset

​Deleting a dataset

Related Guides

Dataset types

Dataset specific fields

Field types

Primitives

Time

Identifier

Geospatial

Arrays

Listing datasets

Creating / Updating a dataset

Accessing a dataset

Deleting a dataset