unit.format seems to be inconsistent?

[cope]

math.unit(0.1, 'W').format({notation: 'fixed'}) gives 0.1 W
math.unit(0.0001, 'W').format({notation: 'fixed'}) gives 100 uW

I would expect either the first one to give 100 mW OR the second one to give 0.1 mW.

In one case is stays at 0.1 while in the other it goes to 100 lower-prefix...

Is there a way to make it consistent?

[simonmilz]

You're absolutely right – the behavior is inconsistent and can be confusing.

If you want consistent units, regardless of how small or large the value is, you can explicitly convert to the desired prefix using .to() before formatting:

math.unit(0.1, 'W').to('mW').format({notation: 'fixed'})
// → "100 mW"

math.unit(0.0001, 'W').to('mW').format({notation: 'fixed'})
// → "0.1 mW"

This way, both values are expressed using the same unit (mW), and you avoid unexpected automatic prefix changes.

[cope]

Right, but unit.format([options]) is supposed to "determine the best fitting prefix for the unit":

[simonmilz]

You're right – according to the docs, unit.format([options]) is meant to determine the best fitting prefix.
But that's exactly why this feels inconsistent:

• 0.1 W stays as 0.1 W (even though 100 mW would arguably be a better fit),
• while 0.0001 W becomes 100 uW (i.e. it does switch to a lower prefix).

So the prefix adjustment kicks in for very small values, but not for medium-small ones – even though 100 mW is arguably a more "pleasant" number than 0.1 W, especially if you're trying to normalize your output.

That’s why I expected both values to either:

• stay in the base unit (W), or
• be converted to a more "readable" prefix (mW/uW), consistently.

Just wanted to point out that this behavior doesn’t fully align with the idea of "best fitting prefix" – at least not in a way that feels predictable.

[gwhitney]

I think saying that the behavior of .format() is "inconsistent" is a bit strong; it's totally accurate, however, to say that the behavior is not fully and clearly documented.

The fact is that the current behavior of .format() deliberately includes a bias toward the "current unit", and it prefers tenths over hundreds. You can demonstrate these rules in the parser window at mathjs.org:

(9000 m).format()
    "9 km"

(900 m).format()
    "900 m"

(0.9 km).format()
    "0.9 km"

(0.09 km).format()
    "90 m"

(900000 mm).format()
    "0.9 km"

(90000 mm).format()
    "90 m"

The behavior is entirely consistent to these rules. The rules may not conform to your judgment of the "best" unit, and they do not have the property that the same measurement will always format to the same string, since the unit that the quantity happens to come in as does affect the output. There is an ongoing discussion at #3418 on how to provide greater control over unit selection, but as you can see there, it is tricky...

[gwhitney]

Have filed #3440 concerning the lack of detail in the Unit documentation.

[cope]

"tenths over hundreds" is perfectly fine

IMO, it should have a bias towards either larger than 1, or less than 1, which ever - so that it always picks 100 vs 0.1 consistently.

IMHO, "a bias toward the current unit", contradicts the idea of determining the best fitting prefix, doesn't it?

[josdejong]

We can try out improving this rule, I agree that I prefer reading 100 over 0.1 too. Anyone interested in creating a PR to see how that works out?

[gwhitney]

I think #3418 is gradually working out an api to control/customize unit selection, which would presumably be usable both in the proposed new .toBest() method and in .format() itself. I believe the API will include the ability to turn off the unit-keeping bias and/or to prefer 100 over 0.1. Given that, it doesn't seem to me that a breaking change is warranted to modify the current unit selection rules, which will become the default in the new scheme.