Protobuf as Client-Side State

Connect-ES makes bindable shapes easy.

Many of my favorite days over the past fifteen years (sheesh, I’m old) were when I worked with Johann on schema management tools for databases.

Now, that’s nearly every day: tools to manage Protobuf schemas are a core product here at Buf. Instead of databases, Protobuf schemas manage your data as it moves over the wire between systems: think browsers and APIs or streams and consumers.

With the release of Connect-ES 2.0, Protobuf’s reach extends beyond the surface of your browser and into popular JavaScript frameworks.

Code for the example below is available in GitHub.

Without Burying the Lead…

Buf’s (free!) CLI, makes it easy to generate code you can immediately use for responsive, bindable state in Svelte, React, or ${JavaScript framework of the week}.

Building on the Todo list from a prior entry, it means we can get rid of our ad-hoc interface representing the Todo shape:

interface Todo {
    text: string
    id: number
    isFinished: boolean
}

Instead of keeping the “shape” of data within your app in your head or just loosely documenting it with “uh, read the code,” going schema first means you can version control the shapes that’ll be shared between server and client, using the Buf CLI to generate code in your local environment or CI/CD pipeline.

(Buf also handles linting, breaking change detection, and other niceties.)

Upgrading Todos to (Proto)Buf

Protobuf may seem dense, but if you can write a class or a CREATE TABLE statement, it’s no big deal. It’s just another language to define shapes of data.

You define your shapes in .proto files, generate code from those, and then use it.

(We’ll talk about services/RPCs/API calls….later.)

Generating our JavaScript

To upgrade our Todo list, I’ll start by defining what a “Todo list” is within a todo.proto file.

// Protobuf "messages" are custom data types, like value objects or DTOs
message TodoList {
  // Proto fields have "tags," which are integers assigning them their
  // positions within binary packets.
  //
  // Think of it as a column position in a database table: if you change
  // the shape of the message, you don't re-use it!
  string name = 1;

  // Arrays can be represented by stating that a field is "repeated",
  // and all of your other messages are available as types
  repeated Todo todos = 2;
}

Next, we’ll define a Todo:

message Todo {
  // In proto, it's important to consider the size
  // of a number and whether or not it's signed 
  // (positive or negative). If all you need are 
  // positive numbers, uint32 or uint64 are solid 
  // bets.
  uint32 id = 1;

  // Strings, though, are just "string"!
  string text = 2;

  // Snake case is the proto convention, and "bool" should make sense.
  bool is_finished = 3;
}

With some .proto source code written, I can run a few Buf CLI commands. Based on my buf.yaml and buf.gen.yaml files, they’ll lint my todo.proto file and generate JavaScript (or Java, or C++, or Python, or any number of available languages):

$ buf lint // shows me no output, because my code is clean!
$ buf generate // no errors!

Over in my IDE, I can see buf generate has created two files based on my .proto:

If you crawl through them, you’ll see they define JavaScript shapes and TypeScript types based on the .proto source:

export declare type Todo = Message<"simplewins.todo.v1.Todo"> & {
  id: number;
  text: string;
  isFinished: boolean;
};

Using our Generated JavaScript

When I first used Connect-ES, I expected to be able to use ES6 classes:

const todo = new Todo()

That’s…problematic. ES6 classes don’t serialize well to JSON, making them hard to use within modern JavaScript frameworks.

Connect-ES 2.0 represents data as plain JavaScript objects: they work great with frameworks.

To work with generated shapes, we’ll use the create() function from @bufbuild/protobuf:

Instead of:

const todo = {text: text, isFinished: false, id: this.nextId++}

We now use create() to insist that what we’re creating is linked to a schema:

const todo = create(TodoSchema, {text: text, isFinished: false, id: this.nextId++})

That’s it.

I’ll admit it seems minor in this example. In a work project, I enjoyed having ready-made data types within my SvelteKit application without extracting JavaScript shapes into TypeScript type definitions.

Still, I wasn’t excited until I could share those shapes with the server side — that’s when Protobuf and the Buf workflow really came alive.