golang protobuf remove omitempty tag from generated json tags

Question

I am using google grpc with a json proxy. for some reason i need to remove the omitempty tags from the struct generated in the *.pb.go files.

if i have a proto message like this

message Status {
  int32 code = 1;
  string message = 2;
}

The generated struct looks like this

type Status struct {
  Code int32 `protobuf:"varint,1,opt,name=code" json:"code,omitempty"`
  Message string `protobuf:"bytes,2,opt,name=message" json:"message,omitempty"`
}

But My need is to remove the omitempty tag from the generated structs. How can i do this?

Answer 1

If you are using grpc-gateway and you need the default values to be present during json marshaling, you may consider to add the following option when creating your servemux

    gwmux := runtime.NewServeMux(runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONPb{OrigName: true, EmitDefaults: true}))

Outside of grpc-gateway, if you want to marshal your protocol buffer message, use google.golang.org/protobuf/encoding/protojson (*) package instead of encoding/json

func sendProtoMessage(resp proto.Message, w http.ResponseWriter) {
    w.Header().Set("Content-Type", "application/json; charset=utf-8")
    m := protojson.Marshaler{EmitDefaults: true}
    m.Marshal(w, resp) // You should check for errors here
}

(*) google.golang.org/protobuf replaces the now deprecated github.com/golang/protobuf and its jsonpb package.

Answer 2

A [more] portable solution:

Use sed to strip the tags after generating via protoc.

Example of what I actually use in my go:generate script after having generated the *.pb.go files:

ls *.pb.go | xargs -n1 -IX bash -c 'sed s/,omitempty// X > X.tmp && mv X{.tmp,}'

---

Note: sed -i (inline-replacement) is not used here because that flag isn't portable between standard OS-X and Linux.

Answer 3

I found that the omitempty json tag is hard-coded into the protoc-gen-go source around line 1778:

tag := fmt.Sprintf("protobuf:%s json:%q",
    g.goTag(message, field, wiretype), jsonName+",omitempty")

it will be easy changing the source and make a new protoc-gen-go binary yourself.

It's worth noting that this is likely inadvisable and not recommended for several reasons, particularly because you will then be responsible for ensuring that the hacked up binary always gets used if the protobufs need to be regenerated.

Answer 4

You can try using gogo proto (https://github.com/gogo/protobuf) With the jsontag extension, your proto message would look like

message Status {
  int32 code = 1 [(gogoproto.jsontag) = "code"];
  string message = 2 [(gogoproto.jsontag) = "message"];
}

You can also add more tags, if you like.

Answer 5

I am posting an update to DeeSilence's answer that works with the latest protobuf version (at the moment of writing).

import "google.golang.org/protobuf/encoding/protojson"

m := protojson.MarshalOptions{EmitUnpopulated: true}
resp, err := m.Marshal(w)

Answer 6

The Marshaler under the jsonpb package has a EmitDefaults field. Setting this to true, will just ignore the omitempty tag in struct.

https://godoc.org/github.com/golang/protobuf/jsonpb#JSONPBMarshaler

Answer 7

You could use "sed" command to remove this text from files like following

sed -i "" -e "s/,omitempty//g" ./api/proto/*.go

where args:

1. -i "" is meaning that keep same name of files
2. -e "s/,omitempty//g" = format to replace like "s/SEARCH/INSERT/g"

Answer 8

To expand on @colm.anseo's edit saying that the google.golang.org/protobuf/encoding/protojson is now deprecated and now google.golang.org/protobuf should be used, here is an example how to do it without modifying the proto.

import (
    "google.golang.org/protobuf/encoding/protojson"
)

func main() {
    someProto := SomeProto{}
    marshaler := protojson.MarshalOptions{EmitUnpopulated: true}
    output, _ := marshaler.Marshal(someProto)
    ... 
}

This is the only way to do it if you don't want to modify the generated struct / are importing the struct via buf and the bsr.

Answer 9

you can copy the encoding/json package to your own folder for example my_json, and modify omitEmpty field to false, and use my_json.Marshal() to encode the struct to json string.