encoding/binary: add var NativeEndian; also x/sys/cpu.IsBigEndian

[bradfitz]

I'd like to revisit #35398 (proposal: encoding/binary: add NativeEndian). Sorry. I know the history & people's opinions already. But I come with a story! 😄

Go 1.19 added support forGOARCH=loong64 (https://go.dev/doc/go1.19#loong64)

So naturally somebody wanted to compile tailscale.com/cmd/tailscaled with GOOS=linux GOARCH=loong64. Compilation failed.

It turned out we have four different native endian packages in our dependency tree:

• https://pkg.go.dev/tailscale.com/util/endian (const Big = false, var Native = binary.LittleEndian)
• https://pkg.go.dev/github.com/josharian/native (var Endian = binary.LittleEndian)
  • example dep path: "tailscale.com/net/interfaces" => "github.com/mdlayher/netlink" => "github.com/mdlayher/netlink/nlenc" "github.com/josharian/native"
• https://pkg.go.dev/github.com/u-root/uio/ubinary (var NativeEndian = binary.LittleEndian)
  • example dep path: "tailscale.com/wgengine/router" => "tailscale.com/net/dns" => "tailscale.com/net/tstun" => "github.com/insomniacslk/dhcp/dhcpv4" => "github.com/u-root/uio/uio" => "github.com/u-root/uio/ubinary"
• wireguard-go's upcoming wireguard/endian (in review at https://github.com/WireGuard/wireguard-go/pull/64/files#diff-8b7f475f3cbc65d1d51dd1c959d0232aad2aed0fbf7967207a97bfb0f94abdba) because it doesn't take external dependencies outside of std or golang.org/x/*

So we had to update all four, along with the various requisite go.mod bumps.

Some observations:

• they're all very similar
• people don't like taking dependencies on other big packages. @josharian's github.com/josharian/native that he mentioned in proposal: encoding/binary: add NativeEndian #35398 (comment) is closest, but lacks the constant that we ended up needing in Tailscale. So everybody makes their own local copies instead. That works until a new GOARCH comes along. Maybe that's rare enough? But I'm sure more riscv* variants will come along at some point.

x/sys/cpu already has this code:

https://cs.opensource.google/go/x/sys/+/refs/tags/v0.3.0:cpu/byteorder.go;l=44

And it has even more GOARCH values (for gccgo) than any other package has!

So everybody has a different subset of GOARCH values it seems.

I know people don't want to encourage thinking about or abusing endianness, but it's a reality when talking to kernel APIs. And this is kinda ridiculous, having this duplicated incompletely everywhere.

It would've been neat if Go could've added loong64 and had a bunch of code in the Go ecosystem just work right away and not require adjusting build tags.

Alternatively, if std and x/sys/cpu are too objectionable: what about new build tags?

/cc @josharian @mdlayher @hugelgupf @zx2c4 @yetist @jwhited @raggi

[mdlayher]

Yes please. Another option: unsafe.NativeEndian to imply that you shouldn't use this unless you are aware of the implications?

[ianlancetaylor]

CC @robpike

[josharian]

@josharian's github.com/josharian/native that he mentioned in #35398 (comment) is closest, but lacks the constant that we ended up needing in Tailscale.

FWIW, I know the author of that package, and he’s amenable to adding a constant.

But an x/sys/cpu or build tag option would be better. :)