Protocol Buffers Data Type Go Code Generation
External
https://protobuf.dev/reference/go/go-generated
Internal
Overview
Protocol Buffer files are processed by the Protocol Buffer compiler protoc, which needs the Go plugin protoc-gen-go to generate Go code. The plugin is installed as described in the Installation section below. The protoc command line details are discussed in the Code Generation section.
Installation
The generic compiler must be installed with:
brew install protobuf
To generate Go code, the Go plugin must be installed with:
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
This will install a protoc-gen-go binary in ${GOBIN}.
Code Generation
The compiler requires a mandatory source directory, specified with --proto_path, where it expects the find the .proto Protocol Buffer files, possibly scattered over a directory sub-hierarchy. The Protocol Buffer files are individually specified on the compiler's command line, and their paths must be relative to --proto_path.
The compiler creates one single Go source file for each .proto input file. The name of the output file is the name of the corresponding .proto file where the .proto extension is replaced with .pb.go. For example, a person.proto Protocol Buffer file will generate a person.pb.go Go source file.
The compiler must know Go package's import path for every .proto it processes, including those transitively depended upon by the .proto files being processed. Is the "import path" mentioned here the import path for the generated Go package, or something else? Is there an "import path" concept for the Protocol Buffer files themselves? There are two ways to specify the Go package import path:
- By declaring it within the
.protofile withoption go_package. This method is recommended and described in detail in the Declare Package Input Path inside the .proto Files section. - By declaring it on the command line when invoking
protocusing--go_optoptions.
The newly created Go source files will be placed somewhere in the root directory specified by --go_out, but where exactly depends on the option go_package setting embedded within the file or on the --go_opt compiler flags, as shown below.
protoc \
--proto_path=<protocol-buffer-root-source-directory> \
--go_out=<protocol-buffer-root-generated-code-directory> \
[--go_opt=...] \
.../<protocol-buffer-file-1>.proto .../<protocol-buffer-file-1>.proto ...
For more details on a typical Go project layout, see:
Declare Package Input Path inside the .proto Files
The Protocol Buffer compiler must know the path of the directory where to place the generated .pb.go file, relative to the --go_out directory. The recommended way to specify this path is to use the go_package option inside the .proto file. The value of the go_package option is the path of the directory, relative to the --go_out directory, where the generated file will be placed.
syntax = "proto3";
option go_package = "./somepkgpb"; // the compiler will create "./somepkgpb" in the directory indicated by --go_out
message Something {
...
}
compiled with:
protoc ... --go_out=./pkg somefile.proto
will make the compiler to place the generated somefile.pb.go in ./pkg/somepkgpb. Note that the compiler expects the go_package option value to be a filesystem path, so it expects to see at least a dot (".") or a slash ("/").
The package name for the generated code file will be by default automatically inferred as the the last path fragment of the path specified with go_package option. In the example above, the package name will be somepkgpb.
An explicit package name may be specified, by separating the name from the import path by a semicolon ("./somepkgpb;someothername"). However, this usage is discouraged since the package name will be derived by default from the import path in a reasonable manner.
Package names should follow Protocol Buffer package name convention.
Package Naming Convention
The package name for the Go source code files generated from .proto files are conventionally suffixed with "pb": somepkgpb. Even if it not required to use the same name for the enclosing directory, it is good practice if that happens.
Example
A typical layout that includes Protocol Buffer files that will generate code in different packages:
.
├── go.mod
├── pkg
│ ├── main
│ │ └── main.go
│ ├── somepkgpb ┌─────────────────────────────────────────┐
│ │ ├── file1.pb.go ─┤ package somepkgpb │
│ │ └── file2.pb.go └─────────────────────────────────────────┘
│ └── someotherpkgpb ┌─────────────────────────────────────────┐
│ ├── file3.pb.go ─┤ package someotherpkgpb │
│ └── file4.pb.go └─────────────────────────────────────────┘
└── protobuf
├── somepkgpb ┌─────────────────────────────────────────┐
│ ├── file1.proto ─┤ option go_package = "./somepkgpb"; │
│ └── file2.proto │ package somepkgpb; │
│ └─────────────────────────────────────────┘
└── someotherpkgpb ┌─────────────────────────────────────────┐
├── file3.proto ─┤ option go_package = "./someotherpkgpb"; │
└── file4.proto │ package someotherpkgpb; │
└─────────────────────────────────────────┘
The protoc command is:
protoc \
--proto_path=./protobuf --go_out=./pkg \
somepkgpb/file1.proto somepkgpb/file2.proto someotherpkgpb/file3.proto someotherpkgpb/file4.proto
Compilation
Update the necessary dependencies to go.mod with:
go get google.golang.org/protobuf/reflect/protoreflect
--go_opt
paths=import
TODO.
paths=source_relative
TODO.
module=${PREFIX}
TODO.
M${PROTO_FILE}=${GO_IMPORT_PATH}
TODO. Use Declare Package Input Path inside the .proto Files instead.
Generated Code
Translation rules:
- The dots in the Protocol Buffer package name are translated to underscore in the Go package name.
- The generated fields use a CamelCase convention, translated from snake_case.
- "protobuf" and "json" tags are also generated in the struct.
The Protocol Buffer source message:
option go_package = "./somepkgpb";
package somepkgpb;
message SomeMessage {
int32 id = 1;
bool is_valid = 2;
string name = 3;
repeated string sample_list = 4;
}
The Go file:
package somepkgpb
type SomeMessage struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id int32 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
IsValid bool `protobuf:"varint,2,opt,name=is_valid,json=isValid,proto3" json:"is_valid,omitempty"`
Name string `protobuf:"bytes,3,opt,name=name,proto3" json:"name,omitempty"`
SampleList []string `protobuf:"bytes,4,rep,name=sample_list,json=sampleList,proto3" json:"sample_list,omitempty"`
}
func (x *SomeMessage) Reset() { ... }
func (x *SomeMessage) String() string { ... }
func (*SomeMessage) ProtoMessage() {}
func (x *SomeMessage) ProtoReflect() protoreflect.Message { ... }
func (x *SomeMessage) GetId() int32 { ... }
func (x *SomeMessage) GetIsValid() bool { ... }
func (x *SomeMessage) GetName() string { ... }
func (x *SomeMessage) GetSampleList() []string { ... }
Generated enums
syntax = "proto3";
option go_package = "./somepkgpb";
package somepkgpb;
enum SomeEnum {
UNKNOWN = 0;
A = 1;
B = 2;
C = 3;
}
generates:
package somepkgpb
type SomeEnum int32
const (
SomeEnum_UNKNOWN SomeEnum = 0
SomeEnum_A SomeEnum = 1
SomeEnum_B SomeEnum = 2
SomeEnum_C SomeEnum = 3
)
// Enum value maps for SomeEnum.
var (
SomeEnum_name = map[int32]string{
0: "UNKNOWN",
1: "A",
2: "B",
3: "C",
}
SomeEnum_value = map[string]int32{
"UNKNOWN": 0,
"A": 1,
"B": 2,
"C": 3,
}
)
For more details about enums, see: