Skip to main content

Step 3: Convert JSON to Protobuf

Protocol Buffers (Protobuf) is another common binary format, widely used in high-performance microservice communication, especially with gRPC. Like Avro, it is schema-based, compact, and efficient.

This step teaches you how to convert a JSON message to the Protobuf format.

The Goal​

You will create a pipeline that takes a JSON message, validates it against a Protobuf schema, and converts it to the binary Protobuf format.

The "Prepare -> Convert" Pattern​

  1. Define Schema: Protobuf requires a schema in a .proto file that describes the message structure.
  2. Prepare: The incoming JSON must be structured to match the .proto definition.
  3. Convert: Supply an encoder compatible with the message schema and consumer framing.

Implementation​

  1. Define the Protobuf Schema: Create a file named sensor.proto. This schema defines the structure for our sensor data.

    sensor.proto
    syntax = "proto3";

    message SensorReading {
      string sensor_id = 1;
      double temperature = 2;
      int64 timestamp_unix_ms = 3;
    }

  2. Implementation boundary: The schema above illustrates the desired record shape. The maintained complete configuration converts JSON, CSV, and XML; it does not include this Protobuf encoder. Supply and validate that integration separately.

Verification​

Decode the output with the matching Protobuf schema and compare fields and types. Binary console output alone does not establish correct encoding or gRPC interoperability. This page has no current executed Protobuf result.