> ## Documentation Index > Fetch the complete documentation index at: https://docs.tilebox.com/llms.txt > Use this file to discover all available pages before exploring further. # Protobuf > Understand how Protocol Buffers provide strongly typed data structures for working with Tilebox datasets in Go, and how to generate them for your projects. Tilebox uses [Protocol Buffers](https://protobuf.dev/), with a custom generation tool, combined with standard Go data structures. [Protocol Buffers](https://protobuf.dev/) (often referred to as `protobuf`) is a schema definition language with an efficient binary format and native language support for lots of languages, including Go. Protocol buffers are open source since 2008 and are maintained by Google. ## tilebox-generate Protobuf schemas are typically defined in a `.proto` file, and then converted to a native Go struct using the protobuf compiler. Tilebox datasets already define a protobuf schema as well, and automate the generation of Go structs for existing datasets through a quick `tilebox-generate` command-line tool. See [Installation](/sdks/go/install) for more details on how to install `tilebox-generate`. ```sh theme={"system"} tilebox-generate --dataset open_data.copernicus.sentinel1_sar ``` The preceding command will generate a `./protogen/tilebox/v1/sentinel1_sar.pb.go` file. More flags can be set to change the default output folders, package name, etc. This file contains everything needed to work with the [Sentinel-1 SAR](https://console.tilebox.com/datasets/explorer/e27e6a58-c149-4379-9fdf-9d43903cba74) dataset. It's recommended to check the generated files you use in your version control system. If you open this file, you will see that it starts with `// Code generated by protoc-gen-go. DO NOT EDIT.`. It means that the file was generated by the `protoc-gen-go` tool, which is part of the protobuf compiler. After editing a dataset, you can call the generate command again to ensure that the changes are reflected in the generated file. The file contains a `Sentinel1Sar` struct, which is a Go struct that represents a datapoint in the dataset. ```go Go theme={"system"} type Sentinel1Sar struct { xxx_hidden_GranuleName *string `protobuf:"bytes,1,opt,name=granule_name,json=granuleName"` xxx_hidden_ProcessingLevel v1.ProcessingLevel `protobuf:"varint,2,opt,name=processing_level,json=processingLevel,enum=datasets.v1.ProcessingLevel"` // more fields } ``` Notice that the fields are private (starting with a lowercase letter), so they are not accessible. Protobuf hides the fields and provides getters and setters to access them. ## Protobuf 101 ### Initializing a message Here is how to initialize a `v1.Sentinel1Sar` message. ```go Go theme={"system"} import ( "google.golang.org/protobuf/proto" "google.golang.org/protobuf/types/known/timestamppb" ) datapoint := v1.Sentinel1Sar_builder{ Time: timestamppb.New(time.Now()), GranuleName: proto.String("S1A_EW_GRDH_1SSH_20141004T020507_20141004T020611_002673_002FAF_8645_COG.SAFE"), ProductType: proto.String("EW_GRDH_1S-COG"), FileSize: proto.Int64(488383473), }.Build() ``` Protobuf fields are private and provides a builder pattern to create a message. `proto.String` is a helper function that converts `string` to `*string`. This allows protobuf to differentiate between a field that is set to an empty string and a field that is not set (nil). An exhaustive list of those helper functions can be found [here](https://github.com/golang/protobuf/blob/master/proto/wrappers.go). Only primitives have a `proto.XXX` helper function. Complex types such as timestamps, durations, UUIDs, and geometries have a [constructor function](#constructors). ### Getters and setters Protobuf provides methods to get, set, clear and check if a field is set. ```go Go theme={"system"} fmt.Println(datapoint.GetGranuleName()) datapoint.SetGranuleName("my amazing granule") datapoint.ClearGranuleName() if datapoint.HasGranuleName() { fmt.Println("Granule name is set") } ``` Getters for primitive types will return a Go native type (for example, int64, string, etc.). Getters for complex types such as timestamps, durations, UUIDs, and geometries can also be converted to more standard types using [AsXXX](#asxxx-methods) methods. ## Well known types Beside Go primitives, Tilebox supports some well known types: * Duration: A duration of time. See [Duration](https://protobuf.dev/reference/protobuf/google.protobuf/#duration) for more information. * Timestamp: A point in time. See [Timestamp](https://protobuf.dev/reference/protobuf/google.protobuf/#timestamp) for more information. * UUID: A [universally unique identifier (UUID)](https://en.wikipedia.org/wiki/Universally_unique_identifier). * Geometry: Geospatial geometries of type Point, LineString, Polygon or MultiPolygon. They have a couple of useful methods to work with them. ### Constructors ```go Go theme={"system"} import ( "github.com/paulmach/orb" datasetsv1 "github.com/tilebox/tilebox-go/protogen/datasets/v1" "google.golang.org/protobuf/types/known/durationpb" "google.golang.org/protobuf/types/known/timestamppb" ) timestamppb.New(time.Now()) durationpb.New(10 * time.Second) datasetsv1.NewUUID(uuid.New()) datasetsv1.NewGeometry(orb.Point{1, 2}) ``` ### `CheckValid` method `CheckValid` returns an error if the field is invalid. ```go Go theme={"system"} err := datapoint.GetTime().CheckValid() if err != nil { fmt.Println(err) } ``` ### `IsValid` method `IsValid` reports whether the field is valid. It's equivalent to `CheckValid == nil`. ```go Go theme={"system"} if datapoint.GetTime().IsValid() { fmt.Println("Valid") } ``` ### `AsXXX` methods `AsXXX` methods convert the field to a more user friendly type. * `AsUUID` will convert a `datasetsv1.UUID` field to a [uuid.UUID](https://pkg.go.dev/github.com/google/uuid#UUID) type * `AsTime` will convert a `timestamppb.Timestamp` field to a [time.Time](https://pkg.go.dev/time#Time) type * `AsDuration` will convert a `durationpb.Duration` field to a [time.Duration](https://pkg.go.dev/time#Duration) type * `AsGeometry` will convert a `datasetsv1.Geometry` field to an [orb.Geometry](https://github.com/paulmach/orb?tab=readme-ov-file#shared-geometry-interface) interface ```go Go theme={"system"} datapoint.GetId().AsUUID() // uuid.UUID datapoint.GetTime().AsTime() // time.Time datapoint.GetDuration().AsDuration() // time.Duration datapoint.GetGeometry().AsGeometry() // orb.Geometry ``` Those methods performs conversion on a best-effort basis. Type validity must be checked beforehand using `IsValid` or `CheckValid` methods. ## Common data operations Datapoints are contained in a standard Go slice so all the usual [slice operations](https://gobyexample.com/slices) and [slice functions](https://pkg.go.dev/slices) can be used. The usual pattern to iterate over data in Go is by using a `for` loop. As an example, here is how to extract the `copernicus_id` fields from the datapoints. ```go Go theme={"system"} // assuming datapoints has been filled using `client.Datapoints.QueryInto` method var datapoints []*v1.Sentinel1Sar copernicusIDs := make([]uuid.UUID, len(datapoints)) for i, dp := range datapoints { copernicusIDs[i] = dp.GetCopernicusId().AsUUID() } ``` Here is an example of filtering out datapoints that have been published before January 2000 and are not from the Sentinel-1C platform. ```go Go theme={"system"} jan2000 := time.Date(2000, time.January, 1, 0, 0, 0, 0, time.UTC) // slice of length of 0, but preallocate a capacity of len(datapoints) s1cDatapoints := make([]*v1.Sentinel1Sar, 0, len(datapoints)) for _, dp := range datapoints { if dp.GetPublished().AsTime().Before(jan2000) { continue } if dp.GetPlatform() != "S1C" { continue } s1cDatapoints = append(s1cDatapoints, proto.CloneOf(dp)) // Copy the underlying data } ``` ## Converting to JSON Protobuf messages can be converted to JSON without loss of information. This is useful for interoperability with other systems that doesn't use protobuf. A guide on protoJSON can be found format here: [https://protobuf.dev/programming-guides/json/](https://protobuf.dev/programming-guides/json/) ```go Go theme={"system"} originalDatapoint := datapoints[0] // Convert proto.Message to JSON as bytes jsonDatapoint, err := protojson.Marshal(originalDatapoint) if err != nil { log.Fatalf("Failed to marshal datapoint: %v", err) } fmt.Println(string(jsonDatapoint)) ``` ```plaintext Output theme={"system"} {"time":"2001-01-01T00:00:00Z","id":{"uuid":"AOPHpzQAAmV2MZ4+Zv+JGg=="},"ingestionTime":"2025-03-25T10:26:10.577385176Z","granuleName":"MCD12Q1.A2001001.h02v08.061.2022146033342.hdf","geometry":{"wkb":"AQMAAAABAAAABQAAAFIi9vf7TmTAXsX3////I0Bexff///9jwAAAAAAAAAAACUn4//+/YsAAAAAAAAAAAC7AdjgMCmPAXsX3////I0BSIvb3+05kwF7F9////yNA"},"endTime":"2001-12-31T23:59:59Z","horizontalTileNumber":"2","verticalTileNumber":"8","tileId":"51002008","fileSize":"176215","checksum":"771212892","checksumType":"CKSUM","dayNightFlag":"Day","publishedAt":"2022-06-23T10:58:13.895Z"} ``` It can also be converted back to a `proto.Message`. ```go Go theme={"system"} // Convert JSON bytes to proto.Message unmarshalledDatapoint := &v1.Sentinel1Sar{} err = protojson.Unmarshal(jsonDatapoint, unmarshalledDatapoint) if err != nil { log.Fatalf("Failed to unmarshal datapoint: %v", err) } fmt.Println("Are both equal?", proto.Equal(unmarshalledDatapoint, originalDatapoint)) ``` ```plaintext Output theme={"system"} Are both equal? true ```