cmd/go: add *_unix.go recognition (and presumably unix build tag)

[robpike]

It does understand *_windows.go, and I was surprised to (re)learn that it does not support *_unix.go. Would it do so, things would be clearer and all the *_unix.go files that already exist wouldn't need to maintain their

// +build darwin dragonfly freebsd linux netbsd openbsd solaris

tags as the list grows.

[bradfitz]

More concretely, does this mean "unix" would mean anything in a pre-defined list of GOOS values? Would we proactively add unsupported GOOS values that might be supported later, like we did with GOARCH values, or would "unix" mean anything that's not windows or plan9 or nacl? Wait, is nacl unix?

Also, should we define "posix"? Currently we have some _posix.go files which include windows.

I don't object to this, but I'm just curious which definition we'd use.

[robpike]

All good questions for which I don't know the answer.

My main concern is maintaining the long list of os names in the build tags. Making things easier would be an advantage too.

Discussion needed.

[j7b]

I'd suggest the focus should be on user code and the standard library, and a new build tag is required (perhaps "unix"). The standard library has clear manifestations of a notion of "unix" in packages intended for general use (most obviously in net), and I'd suggest any GOOS that implements these should satisfy the build constraint word "unix." For example, if nacl's net package supports network types "unix", "unixgram" and "unixpacket" it would satisfy some of the predicates the "unix" build constraint should require.

The *_unix.go convention could implicitly define +build unix.

[robpike]

That's pretty much what I mean.

[dominikh]

Related: #6325

[jimmyfrasche]

Looking at the list of OSes, the BSD's may be the most homogeneous and largest sublist. But do you count Solaris, Darwin? Even if not, just defining bsd as dragonfly freebsd netbsd openbsd seems handy regardless of any other tags that might be included. If Darwin and Solaris are included, the build tag in the OP shrinks down to // +build linux bsd

Looking at the feature sets, posix may be the simplest target, but which subset of posix? Is it just the shared subset of all the unix-like OSes currently supported? When a new posix-y OS is added does that mean it can't be added to posix if it's missing one thing the other posix OSes support or do the set of guaranteed features lose a member? Since posix is so vast and as far as I know in practice little agreed upon, that seems the most problematic.

Looking at the stdlib, there seems to be very little agreement in practice among which OSes to include, even in *_unix.go and *_posix.go files. Of course, that doesn't preclude having build tags like // +build unix,!freebsd when needed.

Looking to the future, if the list is not proactive, as Brad put it, that means a new OS could suddenly match with a new version of Go when it should not. That may be worrisome but then again currently we have the situation where a new GOOS doesn't match anything without manual intervention. Of course, even a future proofed list is only as good as targets currently known and can't contain as yet uninvented OSes, and then the same issues arise as with posix.

[j7b]

From a user code perspective it shouldn't matter if the "unix" is bsd, lunix, solaris or other. I'd consider mandating cgo for test purposes for "unix" candidates and having a cgo-centric test suite to make sure all the unix-type-things for a given Go toolchain aren't at odds with the C toolchain as known to the Go toolchain for a given GOOS.

[rsc]

This is basically a straight duplicate of #6325, as @dominikh points out. The problem here, as there, is that "unix" changes based on various details - sometimes it includes Linux, sometimes not, for example. It's not clear we want to be in the business of defining it exactly one way. Also, this should not affect users terribly much, given that we try to provide OS-independent APIs to the largest extent possible.

[robpike]

You're skirting the point here, which is just to avoid maintaining the huge list of Unix variants Go supports. It's to discriminate the basic difference between Windows and Unix. Windows has many versions, but there is only one predefined "windows" tag, in the sense the foo_windows.go works. I'm asking for the same courtesy for Unix.

[jimmyfrasche]

Ignoring ios and android for the moment, as they're essentially defined to be "variants" of darwin and linux, and only looking at the GOOS build tags, the complete list, as of 1.8, is:

windows plan9 nacl linux solaris darwin dragonfly openbsd netbsd freebsd

The goal, as I understand it, is to be able to use *_GOOS.go as much as possible, by defining one or more "super-GOOS" tags that match a large subset of the above tags.

I'm going to ignore the implications of defining unix or posix and just look at what's in the stdlib so there's some data. (Apologies if I made any mistakes).

Looking at the *_posix.go files in the $GOROOT/src† the definition of posix uses seems to be almost equivalent to !plan9, with occasion to additionally specify !nacl (3/15 of the files) and once adding !solaris.

(Windows is not what most people think of when they thing posix. An alternative would be to allow something like x_not-T.go to be equivalent to a // +build !T directive and rename most of those as _not-plan9.go.)

† using

find -type f -name "*_posix.go" -exec grep "^// [+]build " {} \; | cut -d" " -f3- | sort

For the *_unix.go files,I took the list of tags, removed two that, despite the name, only had to do with cgo, took out all arches, android build tags, etc., manually expanded the negations, reordered them so all the tags were in the same order and then counted the occurrences of each unique subset of GOOSes:

18: darwin dragonfly freebsd linux nacl netbsd openbsd solaris
12: darwin dragonfly freebsd linux netbsd openbsd solaris
3: darwin dragonfly freebsd linux nacl netbsd openbsd plan9 solaris
2: darwin dragonfly freebsd linux netbsd openbsd
2: darwin dragonfly freebsd linux nacl netbsd openbsd solaris windows
1: dragonfly freebsd linux nacl netbsd openbsd solaris
1: dragonfly freebsd linux netbsd openbsd
1: dragonfly linux netbsd openbsd solaris
1: freebsd linux netbsd

Looking at that it seems !windows,!plan9 would get the most existing *_unix.go files, fit most people's expected definition, but often require an additional // +build !nacl directive.

In practice how build tags are written seems to be somewhat ad hoc. Maybe gofmt could sort them in a standard way or golint warn if they're not alphabetical? Perhaps a new tool that deals solely with build constraints?

[jimmyfrasche]

Here's similar for every GOOS build tag :
https://gist.github.com/jimmyfrasche/24f45e37a35f02eab8dcd1fb2c1c6af2

[rsc]

It sounds like the real interesting use would be outside the standard library. Can someone survey what usages look like outside, like in the go corpus files?