How to define an optional field in protobuf 3

Question

I need to specify a message with an optional field in protobuf (proto3 syntax). In terms of proto 2 syntax, the message I want to express is something like:

Answer 1

Another way to encode the message you intend is to add another field to track "set" fields:

syntax="proto3";

package qtprotobuf.examples;

message SparseMessage {
    repeated uint32 fieldsUsed = 1;
    bool   attendedParty = 2;
    uint32 numberOfKids  = 3;
    string nickName      = 4;
}

message ExplicitMessage {
    enum PARTY_STATUS {ATTENDED=0; DIDNT_ATTEND=1; DIDNT_ASK=2;};
    PARTY_STATUS attendedParty = 1;
    bool   indicatedKids = 2;
    uint32 numberOfKids  = 3;
    enum NO_NICK_STATUS {HAS_NO_NICKNAME=0; WOULD_NOT_ADMIT_TO_HAVING_HAD_NICKNAME=1;};
    NO_NICK_STATUS noNickStatus = 4;
    string nickName      = 5;
}

This is especially appropriate if there is a large number of fields and only a small number of them have been assigned.

In python, usage would look like this:

import field_enum_example_pb2
m = field_enum_example_pb2.SparseMessage()
m.attendedParty = True
m.fieldsUsed.append(field_enum_example_pb2.SparseMessages.ATTENDEDPARTY_FIELD_NUMBER)

Answer 2

Since protobuf release 3.12, proto3 has experimental support for using the optional keyword (just as in proto2) to give a scalar field presence information.

syntax = "proto3";

message Foo {
    int32 bar = 1;
    optional int32 baz = 2;
}

A has_baz()/hasBaz() method is generated for the optional field above, just as it was in proto2.

Under the hood, protoc effectively treats an optional field as if it were declared using a oneof wrapper, as CyberSnoopy’s answer suggests:

message Foo {
    int32 bar = 1;
    oneof optional_baz {
        int32 baz = 2;
    }
}

If you’ve already used that approach, you'll be able to clean up your message declarations (switch from oneof to optional) once proto3 optional support graduates from experimental status, since the wire format is the same.

You can find the nitty-gritty details about field presence and optional in proto3 in the Application note: Field presence doc.

Pass the --experimental_allow_proto3_optional flag to protoc to use this functionality in release 3.12. The feature announcement says it will be “generally available hopefully in 3.13”.

Nov 2020 Update: The feature is still considered experimental (flag required) in release 3.14. There are signs of progress being made.

Answer 3

To expand on @cybersnoopy 's suggestion here

if you had a .proto file with a message like so:

message Request {
    oneof option {
        int64 option_value = 1;
    }
}

You can make use of the case options provided (java generated code):

So we can now write some code as follows:

Request.OptionCase optionCase = request.getOptionCase();
OptionCase optionNotSet = OPTION_NOT_SET;

if (optionNotSet.equals(optionCase)){
    // value not set
} else {
    // value set
}

Answer 4

you can find if one has been initialized by comparing the references with the default instance:

GRPCContainer container = myGrpcResponseBean.getContainer();
if (container.getDefaultInstanceForType() != container) {
...
}

Answer 5

Another way is that you can use bitmask for each optional field. and set those bits if values are set and reset those bits which values are not set

enum bitsV {
    baz_present = 1; // 0x01
    baz1_present = 2; // 0x02

}
message Foo {
    uint32 bitMask;
    required int32 bar = 1;
    optional int32 baz = 2;
    optional int32 baz1 = 3;
}

On parsing check for value of bitMask.

if (bitMask & baz_present)
    baz is present

if (bitMask & baz1_present)
    baz1 is present

Answer 6

Based on Kenton's answer, a simpler yet working solution looks like:

message Foo {
    oneof optional_baz { // "optional_" prefix here just serves as an indicator, not keyword in proto2
        int32 baz = 1;
    }
}