Protocol Buffer Concepts

Internal

• Protocol Buffers

Overview

The main use case for Protocol Buffers is sharing data across programming languages. Data can be written and serialized in one language, sent over the network and then and deserialized and interpreted in a different programming language, without compatibility problems.

Protocol Buffers comes with the following advantages:

• It allows defining types, and the data is fully typed when exchanged. We know the type of data in transit.
• Data is compressed automatically.
• Serialization/deserialization is efficient.
• Comes with a schema (the .proto) file, which is used to generate code that writes and reads the data.
• Schema supports embedded documentation.
• Schema can evolve over time in a safe manner. The implementation can be maintained to be backward and forward compatible.

One of the disadvantages is that the data is encoded in a binary format, so it can't be visualized with a text editor.

The typical workflow consist in defining the data types, called messages, then automatically generating the data structures to support and validate the data types, in the programming language of choice. In Go, the messages are represented as structs. With the help of a framework like gRPC, which uses Protocol Buffers as default serialization format and native mechanism to exchange data, client and server code can also be automatically generated. This renders Protocol Buffer convenient for use as serialization format for microservices.

The current version of the protocol is 3, released mid 2016.

Message

A message is a data type in Protocol Buffers. One or more messages are declared in .proto text file with the following format:

syntax = "proto3";

message SomeMessage {
  int32 id = 1;
  string name = 2;
  bool validated = 3;
}

A message has fields.

Fields

Each field has a field type, a field name and a field tag

 <field_type> <field_name> = <field_tag>;

Data Type Support Code Generation

The code that supports the data types defined in the .proto files are generated as follows:

Data Evolution with Protocol Buffers

A message is actually a type. "Message" is used probably because the instances of the types defines as such are mainly intended to be sent over the wire.