go/types: don't panic when multiplying infinities with zero

[ALTree]

Found by @josharian with go-fuzz and factored out from #20227 since it has a different root cause.

The following:

var g = 6e886451608i/2 

works in compile:

tmp/sandbox861754843/main.go:3: constant too large: 6e886451608

but crashes go/types:

panic: multiplication of zero with infinity [recovered]
	panic: multiplication of zero with infinity [recovered]
	panic: multiplication of zero with infinity

goroutine 5 [running]:
testing.tRunner.func1(0xc4200de0f0)
	/Users/josh/go/tip/src/testing/testing.go:712 +0x256
panic(0x12b77e0, 0xc42004e9b0)
	/Users/josh/go/tip/src/runtime/panic.go:489 +0x259
go/types.(*Checker).handleBailout(0xc4200e0000, 0xc420073d08)
	/Users/josh/go/tip/src/go/types/check.go:225 +0xa4
panic(0x12b77e0, 0xc42004e9b0)
	/Users/josh/go/tip/src/runtime/panic.go:489 +0x259
math/big.(*Float).Mul(0xc42000eed0, 0xc42000ee40, 0xc42000eea0, 0x13336f8)
	/Users/josh/go/tip/src/math/big/float.go:1581 +0x11b
go/constant.BinaryOp(0x146a7a0, 0xc42000ee40, 0xe, 0x146a7e0, 0x13336f8, 0x146a7e0, 0x14a5e20)
	/Users/josh/go/tip/src/go/constant/value.go:1056 +0x360
go/constant.mul(0x146a7a0, 0xc42000ee40, 0x146a7e0, 0x13336f8, 0x146a7e0, 0x14a5e20)
	/Users/josh/go/tip/src/go/constant/value.go:1116 +0x52
go/constant.BinaryOp(0x146a760, 0xc42000a4a0, 0xf, 0x146a7e0, 0xc420014478, 0x1170100, 0x1469a20)
	/Users/josh/go/tip/src/go/constant/value.go:1089 +0x1363
go/types.(*Checker).binary(0xc4200e0000, 0xc420058c40, 0xc42000ec30, 0x1469520, 0xc42000a320, 0x1469520, 0xc42000a340, 0xf)
	/Users/josh/go/tip/src/go/types/expr.go:817 +0x510
go/types.(*Checker).exprInternal(0xc4200e0000, 0xc420058c40, 0x1469560, 0xc42000ec30, 0x0, 0x0, 0xc42000a420)
	/Users/josh/go/tip/src/go/types/expr.go:1409 +0x1d64
go/types.(*Checker).rawExpr(0xc4200e0000, 0xc420058c40, 0x1469560, 0xc42000ec30, 0x0, 0x0, 0x0)
	/Users/josh/go/tip/src/go/types/expr.go:945 +0x81
go/types.(*Checker).multiExpr(0xc4200e0000, 0xc420058c40, 0x1469560, 0xc42000ec30)
	/Users/josh/go/tip/src/go/types/expr.go:1481 +0x62
go/types.(*Checker).expr(0xc4200e0000, 0xc420058c40, 0x1469560, 0xc42000ec30)
	/Users/josh/go/tip/src/go/types/expr.go:1475 +0x49
go/types.(*Checker).varDecl(0xc4200e0000, 0xc420085810, 0xc42000c0b0, 0x1, 0x1, 0x0, 0x0, 0x1469560, 0xc42000ec30)
	/Users/josh/go/tip/src/go/types/decl.go:166 +0x27c
go/types.(*Checker).objDecl(0xc4200e0000, 0x146c2c0, 0xc420085810, 0x0, 0xc420073c50, 0x0, 0x8)
	/Users/josh/go/tip/src/go/types/decl.go:81 +0x223
go/types.(*Checker).packageObjects(0xc4200e0000, 0xc42000a460, 0x2, 0x2)
	/Users/josh/go/tip/src/go/types/resolver.go:457 +0x105
go/types.(*Checker).checkFiles(0xc4200e0000, 0xc420073e30, 0x1, 0x1, 0x0, 0x0)
	/Users/josh/go/tip/src/go/types/check.go:239 +0xdb
go/types.(*Checker).Files(0xc4200e0000, 0xc420073e30, 0x1, 0x1, 0xc42000ede0, 0x0)
	/Users/josh/go/tip/src/go/types/check.go:230 +0x49
go/types.(*Config).Check(0xc420058c00, 0x12f539e, 0x1, 0xc420058b40, 0xc420073e30, 0x1, 0x1, 0xc4200856d0, 0x0, 0x2, ...)
	/Users/josh/go/tip/src/go/types/api.go:351 +0x1b3
go/types_test.TestIssue16902(0xc4200de0f0)
	/Users/josh/go/tip/src/go/types/sizes_test.go:106 +0x31f
testing.tRunner(0xc4200de0f0, 0x130bd20)
	/Users/josh/go/tip/src/testing/testing.go:747 +0xd0
created by testing.(*T).Run
	/Users/josh/go/tip/src/testing/testing.go:789 +0x2de
exit status 2
FAIL	go/types	0.011s

[gopherbot]

Change https://golang.org/cl/76190 mentions this issue: go/types: do not crash on Inf*0 Float multiplications

[ALTree]

More crashers from a test I wrote for the CL above (later abandoned since I didn't like the fix):

var f1 = 1e1000000000 * 0
var f2 = 0 * 1e1000000000

var c1 = (-1e1000000000 + 3i) * (0 + 3i)
var c2 = (1e1000000000 + 3i) * (3 + 0i)
var c3 = (3 + 1e1000000000i) * (0 + 3i)
var c4 = (3 + -1e1000000000i) * (3 + 0i)

var c5 = (1e1000000000 + 3i) / (0 + 3i)
var c6 = (-1e1000000000 + 3i) / (3 + 0i)
var c7 = (3 + -1e1000000000i) / (0 + 3i)
var c8 = (3 + 1e1000000000i) / (3 + 0i)

Anything that can cause an Inf * 0 multiplication between Float constants will crash go/types.

[griesemer]

Here's another example:

package p

const x = 1e600000000
const _ = x*0    // ok
const _ = x*x*0  // not ok

where the product x*x leads to an infinity internally. A correct solution really needs to recognize any kind of overflow to infinity.

Concretely, in go/constant/value.go, we need to recognize these cases in the internal constructor functions (makeRat, makeFloat, etc.) and (probably) return an unknownVal if we have overflow. Generally unknown vals are "silent" but in these cases we need to also report an error. Ideally, the existing functions returned an error, but that ship has sailed. We could have "signaling unknown vals" and quiet ones (with a signaling one becoming quiet after first use).

I'm going to assign this to me for now. Still thinking...

[rsc]

Unclear this needs to be fixed for Go 1.10. Feel free to punt to Go 1.11 if you want.