`UINT8` doesn't allow `0xff` (255) as value anymore which breaks code

[DerDreschner]

With in the latest releases of Z2M, there must have been a change in the code regarding processing of messages with UINT8. This leads to the situation, that 255 isn't a valid value anymore, although it is valid, and it's been used multiple times (see for example).

I realized that when I implemented this pull request, I didn't get any value regarding the light settings as well as the maximumBrightness attribute, which allows 255 as a valid value as well (it's actually the default value). The meta attribute contains the correct byte values in that case, but I'm not sure if that was after some changes in zigbee-herdsman or not. I'm not the only person who realized the issue. See Koenkk/zigbee2mqtt#28506 for example.

Is this a bug in zigbee-herdsman or do all converters that need to process 0xff as a valid value have to be changed to the raw message type?

[Koenkk]

If I understand correctly, this happens when reading the brightness right? (because setting should still work).

For reference from the spec:

CC: @Nerivec

[DerDreschner]

That's correct. Setting 0xff does work correctly. Just reading is broken.

[Nerivec]

When a variable value is required to designate an unknown, invalid, null, or undefined data value and there
is no obvious data value (e.g. zero), that is within the valid range, then the Non-Value in the data type table
(see 2.6.2.2) MAY be used.

See Table 2-11 (page 98) for details. But basically almost all types have FF (or equivalent) as "non-value", it's a valid value, but signals "undefined".

Should the "default"s in specific tables not be considered non-values even if they are the non-values for the types?
I'm not quite sure what is proper spec here, but we may have to introduce a new param that's passed as option to the BuffaloZcl to expand capabilities?

currentLevel: {ID: 0, type: DataType.UINT8, default: 0xff}, // i.e. "if value is default, use it as plain"
// or
currentLevel: {ID: 0, type: DataType.UINT8, nonValue: undefined}, // i.e. "there is no non-value"
measuredValue: {ID: 0, type: DataType.INT16, nonValue: -0x8000}, // other example with a non-value

It's not quite clear how to process this though (hence the multiple examples above).

• do we use the "default" as an actual value (i.e. NaN/non-value bypass)?
• do we replace NaN/non-value with the "default" when writing?
• does that mean that a value with a "default" that matches the non-value for the type does not have a non-value?
• some attrs have "MS" as "default", which means it's manufacturer specific if I'm not mistaken, not sure how to handle that on top of above...
• have to be careful with the impl., that could quickly make a mess of behaviors

I'll dig into the Silabs firmware code, see if that part is open source to confirm what they do, so we have a base on this from a "certified" source...

Note: related is also the mandatory/optional aspect. We currently error out on all attributes, we shouldn't. This is mostly a lot of data entry though.

[Nerivec]

I dug a little more. There is the base level control, and a "level control for lighting". The latter is described in the Zigbee spec at 3.19, page 307. Both have ID 0x0008, the "for lightning" is "derived". Use of derived clusters is recommended over the generic base.
The base level control cluster has a device-defined range, that defaults to 0x00..0xff range if not specified, with a default of 0xff (it's not indicated as non-value, so I assume this is a "plain" value, as the range is clearly different otherwise - per below).
The level control for lighting cluster has a fixed 0x01..0xfe range, with a default of 0xff that is used to indicate undefined (non-value).

I took a quick look at the Silabs code, it confirms all the above behaviors.

The current ZH result should be correct for most cases (although zero is also prohibited which is not technically handled...), but we'd need a way to differentiate for non-lighting use. However, this is the only case of "derived" I could find in the spec, so, maybe the base just is expected never to be used (assumed to be a backwards-compat scheme from early spec days)? Not sure how we fit that extra handling otherwise since IDs are identical. The PWM cluster (3.20) also derives from the same base level control, but that one has a different ID.

It also appears we do need to handle the specific "default"s in tables as plain values, and not consider them non-values when present and matching the non-values for the types. While the current ZH result ends up correct for this specific issue, the processing is not technically accurate (likely mismatches for other clusters). We need a way to specify these defaults and to ignore non-values as appropriate, something like this for all types:

-        return value === 0xff ? Number.NaN : value;
+        return value === 0xff && valDefault !== value ? Number.NaN : value;

[kirovilya]

for some attributes, 0xff - is a normal value

[Koenkk]

Fixed in #1510