Addition of YANG compliance TypedValue enum variants

[earies]

• Support for a means to describe nodes (containers) that contain the
  YANG presence statement in gNMI Updates
• Support for a means to describe nodes of type empty (leafs) in gNMI Updates

Description:

As of today, gNMI is meant to be a protocol to access and interact with data on
an endpoint somewhat agnostic of the data content and how the content is
described/modeled. There is however a contract of sorts from an origin
perspective that at least describes how to interact with data modeled in YANG
specific to the OpenConfig model set for the origin = openconfig.

As part of the OpenConfig modeling guidelines, only a subset of the YANG
language is recommended with some being more or less forbidden. One such case
is YANG presence containers described
here.

While other undefined origins can define separate rules to abide by, there is
still a common contract and API defintion that must be shared thus we currently
have gaps for the marrying of YANG modeled data to how such content can be
fully described and properly typed within gNMI message structs.

This PR covers 2 such gaps for YANG statements:

• presence "...";
• type empty;

Example usage for YANG modeled presence container:

  update:  {
    path:  {
      elem:  {
        name:  "configuration"
      }
      elem:  {
        name:  "system"
      }
      elem:  {
        name:  "services"
      }
      elem:  {
        name:  "ftp"
    }
    val:  {
      presence_val:  true
    }
  }

And subsequent deletion (non issue for this PR)

  update:  {
    timestamp:  1743611293000000000
    delete:  {
      elem:  {
        name:  "configuration"
      }
      elem:  {
        name:  "system"
      }
      elem:  {
        name:  "services"
      }
      elem:  {
        name:  "ftp"
      }
    }
  }

Example usage for YANG modeled type empty

  update:  {
    path:  {
      elem:  {
        name:  "configuration"
      }
      elem:  {
        name:  "system"
      }
      elem:  {
        name:  "foo"
      }
      elem:  {
        name:  "bar"
    }
    val:  {
      empty_val:  true
    }
  }

And subsequent deletion (non issue for this PR)

  update:  {
    timestamp:  1743611293000000000
    delete:  {
      elem:  {
        name:  "configuration"
      }
      elem:  {
        name:  "system"
      }
      elem:  {
        name:  "foo"
      }
      elem:  {
        name:  "bar"
      }
    }
  }

[dplore]

Reviewed in OC Community meeting on April 3rd 2025. We would like the community to provide more feedback.

Some initial feedback is these new fields should probably be part of a gnmi extension, and not the base proto.

In Google's use case, we utilize JSON IETF format which explicitly encodes null/empty values. In addition, OC YANG models do not define leaves that have use the YANG empty value. (A common use case for empty is instead met using boolean). For these reasons I don't have a strong opinion on the topic beyond using gnmi extension for this change.

[robshakir]

In Google's use case, we utilize JSON IETF format which explicitly encodes null/empty values.

I don't think this is fully correct. Telemetry updates which must be <path, value> use these native TypedValue fields.

In addition, OC YANG models do not define leaves that have use the YANG empty value.

This is correct.

(A common use case for empty is instead met using boolean). For these reasons I don't have a strong opinion on the topic beyond using gnmi extension for this change.

Can you explain how a gNMI extension can be used here? The TypedValue message is deep within a Notification, doesn't this require creating an entire new hierarchy of updates that shadows the existing approach?

[earies]

In Google's use case, we utilize JSON IETF format which explicitly encodes null/empty values.

I don't think this is fully correct. Telemetry updates which must be <path, value> use these native TypedValue fields.

This is actually touching on another subject that is likely in need of further documentation/guidance which comes down to encoding uses in correspondance to gNMI operations (I'll leave most of that for another time)

For set() - what we are likely to see is a predominant use of JSON_IETF and ASCII in reality especially when you think about full "documents". I would question the use of PROTO encoding for set() other than for single values. The motivator for this PR is less about this operation but more on data retrieval - specifically subscribe()

Now for subscribe(), I'll draw a direct driver here when using native YANG models that may use the full YANG 1.0 or 1.1 defined language. The predominant encoding is PROTO so we need distinct TypedValue mappings to YANG semantics here and as Rob rightly points out, Updates encoded within Notifications must carry <path, value>. The cases of presence containers or empty types have no value and it would be incorrect to only define a prefix or a path with no value.

For a delete notification, we are ok in that we can just emit the path (there is no value).

But for an update notification (ON_CHANGE, SAMPLE), we need some value to pack and clients need to handle this

For reference, JSON_IETF encodes as follows:

Presence container

"foo": {}

Type empty

"foo": [null]

(A common use case for empty is instead met using boolean). For these reasons I don't have a strong opinion on the topic beyond using gnmi extension for this change.

Can you explain how a gNMI extension can be used here? The TypedValue message is deep within a Notification, doesn't this require creating an entire new hierarchy of updates that shadows the existing approach?

An extension cannot be used here. We need a means to map appropriate TypedValue datatypes and a ruleset of how to encode.

[ncorran]

From a Cisco perspective we use GNMI with both openconfig models and our vendor models for set and for subscribe oprations and we have both presence containers and empty leaves in our native models.

We support these new changes as proposed and would add implementation support.

[robshakir]

i am supportive of adding new values to typed value to support these leaf types. i also believe that this cannot be (sanely) achieved through an extension.

for the telemetry use case, there needs to be some thought about how presence is handled, imho. since sending something like /a/presence/container -> TypedValue { presence: true } is going to create an update that is being sent for a non-leaf node. this doesn't comply with the requirements of section 3.5 of the gNMI spec:

When aggregation is not permitted by the client or the schema each update message MUST
contain a (key, value) pair - where the key MUST be a path to a single leaf element within
the data tree (encoded according to Section 2.2.2).
The value MUST encode only the value of the leaf specified.

this will not be compatible with existing gnmi collectors. thus, i think an alternate approach (strawperson proposal: a particular metadata leaf -- e.g., /a/presence/container/@present -- could be used to transmit such information.

[earies]

Thx @robshakir

this will not be compatible with existing gnmi collectors. thus, i think an alternate approach (strawperson proposal: a particular metadata leaf -- e.g., /a/presence/container/@present -- could be used to transmit such information.

What TypedValue would you think we'd want to encode that in? an existing or new? If existing a schema unaware collector cannot decipher this potentially from a leaf node that carries type=string that just so happens to encode string_val: "@present"

If we created a new type (can still be of string) then we are at least signaling that this is rather a piece of metadata on the key (e.g. string metadata_val = 15;) with described encoding rules (@present only to start)

For type=empty - would you agree to keep a proposed bool here to signal an empty "value" ?

[earies]

Thx @robshakir

this will not be compatible with existing gnmi collectors. thus, i think an alternate approach (strawperson proposal: a particular metadata leaf -- e.g., /a/presence/container/@present -- could be used to transmit such information.

What TypedValue would you think we'd want to encode that in? an existing or new? If existing a schema unaware collector cannot decipher this potentially from a leaf node that carries type=string that just so happens to encode string_val: "@present"

If we created a new type (can still be of string) then we are at least signaling that this is rather a piece of metadata on the key (e.g. string metadata_val = 15;) with described encoding rules (@present only to start)

For type=empty - would you agree to keep a proposed bool here to signal an empty "value" ?

or.... are you suggesting that we only define a ruleset (spec update) here for encoding keys and use existing bool_val ?

e.g.

• /a/presence/container/@present = true
• /a/container/leaf/@empty = true

False would never be set. Clients must adapt to accomodate that if the Path describes a YANG modeled path that any PathElem name that starts with "@" is to be handled differently with the value just being stubbed out.

The method I mention prior does not change the path encodings but rather relies off distinct types for the value where we have a protocol of what is encoded as metadata...

[robshakir]

brainstorming:

• empty seems clear:
  • such leaves are just a bool that can never be false, so using bool_val seems reasonable to me.
  • such leaves have their own path -- so /a/path/to/a/leaf/of/type/empty = true if the empty is "set" seems reasonable.
• presence seems more tricky.
  • such leaves do not have their own path -- if we use a metadata-type value -- then we could still have someone actually define a metadata field for a leaf called presence itself, which would have a path of /a/path/to/a/presence/container/@presence -- so my proposal is probably not entirely the right one.
  • i'm suggesting here we have a "fake" leaf path (which the schema-unaware collector just thinks is a 'real' leaf) that represents the "presence" flag underneath a container/list.
    • e.g., /a/path/to/a/presence/container/_presence
    • _ here is just used as this isn't a valid initial character for a leaf in the yang spec.
  • i don't necessarily see that there's any need for anything other than a bool_val here -- since our new "leaf" name would need to be specifically interpreted by a schema-aware client that could just translate bool_val: true for this leaf to mean that it should set the presence flag.

WDYT?

[earies]

brainstorming:

* `empty` seems clear:
  
  * such leaves are just a `bool` that can never be `false`, so using `bool_val` seems reasonable to me.
  * such leaves have their own path -- so `/a/path/to/a/leaf/of/type/empty = true` if the `empty` is "set" seems reasonable.

* `presence` seems more tricky.
  
  * such leaves do not have their own path -- if we use a metadata-type value -- then we could still have someone actually define a metadata field for a leaf called `presence` itself, which would have a path of `/a/path/to/a/presence/container/@presence` -- so my proposal is probably not entirely the right one.
  * i'm suggesting here we have a "fake" leaf path (which the schema-unaware collector just thinks is a 'real' leaf) that represents the "presence" flag underneath a `container`/`list`.
    
    * e.g., `/a/path/to/a/presence/container/_presence`
    * `_` here is just used as this isn't a valid initial character for a leaf in the yang spec.
  * i don't necessarily see that there's any need for anything other than a `bool_val` here -- since our new "leaf" name would need to be specifically interpreted by a schema-aware client that could just translate `bool_val: true` for this leaf to mean that it should set the presence flag.

WDYT?

Apologies for taking so long to revert @robshakir

Agreed re: empty

For presence, agree with everything as well - one minor detail is that _ is actually a valid initial character per YANG

6.2.  Identifiers

   Identifiers are used to identify different kinds of YANG items by
   name.  Each identifier starts with an uppercase or lowercase ASCII
   letter or an underscore character, followed by zero or more ASCII
   letters, digits, underscore characters, hyphens, and dots.
   Implementations MUST support identifiers up to 64 characters in
   length and MAY support longer identifiers.  Identifiers are case
   sensitive.  The identifier syntax is formally defined by the rule
   "identifier" in Section 14.  Identifiers can be specified as quoted
   or unquoted strings.

So we just need to pick another invalid YANG char to use here and one that doesn't conflict w/ @ metadata representation

How about # ?

e.g. /a/path/to/a/presence/container/#presence = true

Any other opinions? @ncorran @rgwilton @jsterne @jgcumming

[ncorran]

I think the empty handling is OK.
I think for presence - since the container is holding data (in terms of presence or not) it is like a pseudo-leaf, so I would much prefer this text:

When aggregation is not permitted by the client or the schema each update message MUST
contain a (key, value) pair - where the key MUST be a path to a single leaf element within
the data tree (encoded according to [Section 2.2.2](https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#222-paths)).

to replace 'leaf' with 'data node' rather than literally a yang leaf (so it can cover leaf, p-container, or list-entry as needed). Doesnt that then make it easier to handle?

Internally we have a flag on a container to say whether it is presence or not and another flag to say whether it exists or not (if it is a presence container). It feels like we should be able to attach the same 'empty leaf' solution to the presence container and encode it with a true-only bool.

But if the only option is a fake leaf then we can certainly handle it.

[ROLANDSIHOMBING1]

I Think complite this argumens in Soul Out.