> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tilebox.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Ingesting data
> Walk through the full process of ingesting GeoParquet data into a Tilebox timeseries dataset, from downloading source files to previewing the results.
This guide is also available as a Google Colab notebook. Click here for an interactive version.
This page guides you through the process of ingesting data into a Tilebox dataset. Starting from an existing
dataset available as file in the [GeoParquet](https://geoparquet.org/) format, you'll go through the process of
ingesting that data into Tilebox as a [Timeseries](/datasets/types/timeseries) dataset.
If you have your data in a different format, check out the [Ingesting from common file formats](/guides/datasets/ingest-format) examples of how to ingest it.
## Related documentation
Learn about Tilebox datasets and how to use them.
Learn how to ingest data into a Tilebox dataset.
## Downloading the example dataset
The dataset used in this example is available as a [GeoParquet](https://geoparquet.org/) file. You can download it
from here: [modis\_MCD12Q1.geoparquet](https://storage.googleapis.com/tbx-web-assets-2bad228/docs/data-samples/modis_MCD12Q1.geoparquet).
## Installing the necessary packages
This example uses a couple of python packages for reading parquet files and for visualizing the dataset. Install the
required packages using your preferred package manager. For new projects, Tilebox recommend using [uv](https://docs.astral.sh/uv/).
```bash uv theme={"system"}
uv add tilebox-datasets geopandas lonboard
```
```bash pip theme={"system"}
pip install tilebox-datasets geopandas lonboard
```
```bash poetry theme={"system"}
poetry add tilebox-datasets="*" geopandas="*" lonboard="*"
```
```bash pipenv theme={"system"}
pipenv install tilebox-datasets geopandas lonboard
```
## Reading and previewing the dataset
The dataset is available as a [GeoParquet](https://geoparquet.org/) file. You can read it using the `geopandas.read_parquet` function.
```python Python theme={"system"}
import geopandas as gpd
modis_data = gpd.read_parquet("modis_MCD12Q1.geoparquet")
modis_data.head(5)
```
```plaintext Output theme={"system"}
time end_time granule_name geometry horizontal_tile_number vertical_tile_number tile_id file_size checksum checksum_type day_night_flag browse_granule_id published_at
0 2001-01-01 00:00:00+00:00 2001-12-31 23:59:59+00:00 MCD12Q1.A2001001.h00v08.061.2022146024956.hdf POLYGON ((-180 10, -180 0, -170 0, -172.62252 ... 0 8 51000008 275957 941243048 CKSUM Day None 2022-06-23 10:54:43.824000+00:00
1 2001-01-01 00:00:00+00:00 2001-12-31 23:59:59+00:00 MCD12Q1.A2001001.h00v09.061.2022146024922.hdf POLYGON ((-180 0, -180 -10, -172.62252 -10, -1... 0 9 51000009 285389 3014510714 CKSUM Day None 2022-06-23 10:54:44.697000+00:00
2 2001-01-01 00:00:00+00:00 2001-12-31 23:59:59+00:00 MCD12Q1.A2001001.h00v10.061.2022146032851.hdf POLYGON ((-180 -10, -180 -20, -180 -20, -172.6... 0 10 51000010 358728 2908215698 CKSUM Day None 2022-06-23 10:54:44.669000+00:00
3 2001-01-01 00:00:00+00:00 2001-12-31 23:59:59+00:00 MCD12Q1.A2001001.h01v08.061.2022146025203.hdf POLYGON ((-172.62252 10, -170 0, -160 0, -162.... 1 8 51001008 146979 1397661843 CKSUM Day None 2022-06-23 10:54:44.309000+00:00
4 2001-01-01 00:00:00+00:00 2001-12-31 23:59:59+00:00 MCD12Q1.A2001001.h01v09.061.2022146025902.hdf POLYGON ((-170 0, -172.62252 -10, -162.46826 -... 1 9 51001009 148935 2314263965 CKSUM Day None 2022-06-23 10:54:44.023000+00:00
```
## Exploring it visually
Geopandas comes with a built in explorer to visually explore the dataset.
```python Python theme={"system"}
from lonboard import viz
viz(modis_data, map_kwargs={"show_tooltip": True})
```
## Create a Tilebox dataset
Now you'll create a [Spatio-temporal](/datasets/types/spatiotemporal) dataset with the same schema as the given MODIS dataset.
To do so, you'll use the [Tilebox Console](/console), navigate to `My Datasets` and click `Create Dataset`. Then select
`Spatio-temporal Dataset` as the dataset type.
For more information on creating a dataset, check out the [Creating a dataset](/guides/datasets/create) guide for a
Step by step guide.
Now, to match the given MODIS dataset, you'll specify the following fields:
| Field | Type | Note |
| ------------------------ | --------- | ----------------------------------- |
| `granule_name` | string | MODIS granule name |
| `end_time` | Timestamp | Measurement end time |
| `horizontal_tile_number` | int64 | Horizontal modis tile number (0-35) |
| `vertical_tile_number` | int64 | Vertical modis tile number (0-17) |
| `tile_id` | int64 | Modis Tile ID |
| `file_size` | uint64 | File size of the product in bytes |
| `checksum` | string | Hash checksum of the file |
| `checksum_type` | string | Checksum algorithm (MD5 / CKSUM) |
| `day_night_flag` | int64 | Day / Night / Both |
| `browse_granule_id` | string | Optional granule ID for browsing |
| `published_at` | Timestamp | The time the product was published |
In the console, this will look like the following:
## Access the dataset from Python
Your newly created dataset is now available. You can access it from Python. For this, you'll need to know the dataset slug,
which was assigned automatically based on the specified `code_name`. To find out the slug, navigate to the dataset overview
in the console.
You can now instantiate the dataset client and access the dataset.
```python Python theme={"system"}
from tilebox.datasets import Client
client = Client()
dataset = client.dataset("tilebox.modis") # replace with your dataset slug
```
## Create a collection
Next, you'll create a collection to insert your data into.
```python Python theme={"system"}
collection = dataset.get_or_create_collection("MCD12Q1")
```
## Ingest the data
Now, you'll finally ingest the MODIS data into the collection.
```python Python theme={"system"}
datapoint_ids = collection.ingest(modis_data)
print(f"Successfully ingested {len(datapoint_ids)} datapoints!")
```
```plaintext Output theme={"system"}
Successfully ingested 7245 datapoints!
```
## Query the newly ingested data
You can now query the newly ingested data. You can query a subset of the data for a specific time range.
Since the data is now stored directly in the Tilebox dataset, you can query and access it from anywhere.
```python Python theme={"system"}
from shapely import Polygon
area = Polygon( # area roughly covering the US
((-124.45, 49.19), (-120.88, 29.31), (-66.87, 24.77), (-65.34, 47.84), (-124.45, 49.19)),
)
data = collection.query(
temporal_extent=("2015-01-01", "2020-01-01"),
spatial_extent=area
)
data
```
```plaintext Output theme={"system"}
Size: 28kB
Dimensions: (time: 110)
Coordinates:
* time (time) datetime64[ns] 880B 2015-01-01 ... 2019-01-01
Data variables: (12/14)
id (time)
For more information on accessing and querying data, check out [querying data](/datasets/query/querying-data).
## View the data in the console
You can also view your data in the Console, by navigate to the dataset, selecting the collection and then clicking
on one of the data points.
## Next steps
Congrats. You've successfully ingested data into Tilebox. You can now explore the data in the console and use it for
further processing and analysis.
Learn all about [querying your newly created dataset](https://docs.tilebox.com/datasets/query)
Explore the different dataset types available in Tilebox
Check out a growing number of publicly available open data datasets on Tilebox