Protobuf

Protocol Buffers ( Protobuf Protobuf Protocol Buffers: a schema language and compact binary format underneath gRPC. You define messages once in a .proto file and generate code for many languages. ) is the schema language and binary serialization Serialization Turning in-memory data into bytes that can be sent over a network or saved, and back again on the other side (deserialization). format underneath gRPC. You define your messages and services once in a .proto file .proto file The schema file where you define Protobuf messages and gRPC services. The protoc compiler turns it into typed code. , then run the protoc compiler protoc The Protobuf compiler. It reads a .proto file and generates typed client and server stubs in the languages you need. to generate typed data classes and client/server stubs in whatever languages you need. The same schema drives a Go service, a Python script, and a Node client, none of them hand-write parsing code, and all of them agree on the exact bytes on the wire:

syntax = "proto3";

// package: a namespace for these types, so names don't collide across .proto files
package library;

// service: the set of RPC methods the server exposes (each generates a stub)
service BookService {
  rpc ListBooks    (ListBooksRequest)  returns (ListBooksResponse);
  rpc GetBook      (GetBookRequest)    returns (Book);
  rpc CreateBook   (CreateBookRequest) returns (Book);
  rpc CheckoutBook (CheckoutRequest)   returns (Book);

  // Server-streaming: push an update every time inventory changes
  rpc WatchBooks   (WatchRequest)      returns (stream Book);
}

// message: a data shape — a record of typed fields, each with a wire number
message Book {
  int32  id        = 1;
  string title     = 2;
  bool   available = 3;
  string author    = 4;
}

message CheckoutRequest   { int32 id = 1; }
message GetBookRequest    { int32 id = 1; }
message CreateBookRequest { string title = 1; string author = 2; }
message ListBooksRequest  {}
message ListBooksResponse { repeated Book books = 1; }
message WatchRequest      {}

Field numbers are the wire format

The field numbers Field number The number assigned to each field in a Protobuf message (= 1, = 2). It, not the field name, is what's actually sent on the wire, so numbers must never be reused. (= 1, = 2) are the real identifiers sent over the wire; human-readable field names are entirely stripped from the binary payload. Each field is encoded as its number paired with a value, so decoding is really a lookup by number, not by name. The generated code maps each number back to a field. That has a consequence worth burning into memory:

• Renaming a field is safe: the name only exists in generated code.
• Changing a field's number is a breaking change: old clients and new servers now disagree about which bytes mean what. Never reassign a number without a migration plan.
• Adding new fields and removing old ones are both backward-compatible: unknown fields are skipped, missing fields get defaults.

This is what makes Protobuf payloads so much smaller and faster to parse than JSON: no key names, no quotes, no whitespace, just numbered, typed binary fields.

Exercise

Complete this .proto file. Fill in the keyword that declares a data shape, the wire number for the title field, and the keyword that marks WatchBooks as server-streaming.

syntax = "proto3";
package library;

service BookService {
  rpc GetBook    (GetBookRequest) returns (Book);
  rpc WatchBooks (WatchRequest)   returns (⌄streamrepeatedoptional Book);
}

⌄messageservicestructtype Book {
  int32  id        = 1;
  string title     = ⌄024;
  bool   available = 3;
}