linspace(...)[1] can differ from linspace start value

[c42f]

Demo:

julia> start = 0.10000000000000045
0.10000000000000045

julia> linspace(start, 1)[1] - start
-1.3877787807814457e-17

julia> linspace(start, 1).start - start
0.0

I think the error here is in unsafe_getindex(::LinSpace, ::Integer), which assumes b*a/b == a, which unfortunately isn't true. The above amounts to computing

julia> 0.10000000000000045 * 49.0 / 49.0 - 0.10000000000000045
-1.3877787807814457e-17

Probably needless to say, this is a big problem for numerical code which should be able to assume LinSpace will exactly preserve the start and end values.

[eschnett]

If this is the code

getindex{T}(r::LinSpace{T}, i::Integer) = (checkbounds(r, i); unsafe_getindex(r, i))
unsafe_getindex{T}(r::LinSpace{T}, i::Integer) = convert(T, ((r.len-i)*r.start + (i-1)*r.stop)/r.divisor)

then changing things to

(r.len-i)/r.divisor * r.start + (i-1)/r.divisor * r.end

would address this.

If the current code is deemed accurate enough, then storing 1 / r.divisor instead of divisor could be an additional performance improvement.

[c42f]

I wonder whether it's possible to solve for a floating point _start, _end, and mult such that the more naive formula

(r.len-i)*r.mult * r._start + (i-1)*r.mult * r._end

exactly reproduces the values at the end points. Then you'd get the best of both worlds (at the cost of some additional setup), though a multiply would be required to get start(r) and end(r)

[ringw]

If r.len == 50, then (r.len-i)*r.mult * r.start + (i-1)*r.mult * r.end is incorrect (assuming r.mult = 1.0 / r.divisor and start and end are not adjusted):

julia> 49.0 * (1.0 / 49.0)
0.9999999999999999

julia> 49.0 / 49.0
1.0

I'm not sure of a good way to adjust mult, start, and end.

Using the formula (i == r.len) ? r.end : (r.start + (i-1)*r.incr) (where r.incr = (r.end - r.start) / r.divisor) seems to be faster than @eschnett's suggestion:

julia> let start = 0.10000000000000045, end_ = 1+eps(), len = 10000000,
       divisor = len-1, incr = (end_ - start) / divisor;
       @time [(i == len) ? end_ : (start + (i-1)*incr) for i in 1:len][1]
       end
  0.055571 seconds (2 allocations: 76.294 MB)
julia> let start = 0.10000000000000045, end_ = 1+eps(), len = 10000000,
       divisor = len-1;
       @time [(len-1)/divisor * start + (i-1)/divisor * end_ for i in 1:len][1]
       end
  0.142083 seconds (2 allocations: 76.294 MB)

[c42f]

It's not what I expected, but I get the same results locally (I put the code into functions and primed the JIT to make sure). Even implementing the linearly interpolated version in terms of multiplication (as I was proposing above) seems a bit slower than your version with a branch. It's a bit surprising that the branch really doesn't hurt performance. I checked the assembly and the branch does seem to have ended up in the inner loop, I guess it's just perfectly predictable.

I have a hunch that the linear interpolated version should be better behaved numerically due to potential cancellation error in computing incr = (end_-start)/divisor, but I didn't manage to come up with an example yet.

[ringw]

I guess the JIT could generate a CMOV instead of a branch, but the branch does seem fast enough.
Most of the pathological cases I tested seem fine:

julia> function test_incr(start, end_, len)
       incr = (end_ - start) / (len - 1)
       return abs(start + incr * (len-1) - end_)
       end
test_incr (generic function with 1 method)

julia> test_incr(0.10000000000000045, 1, 50)
0.0

julia> test_incr(0.10000000000000045, 1, 1e15)
0.0

julia> test_incr(1, 1e15, 100)
0.125

julia> test_incr(1e15, 1e15 + 1, 1000)
0.0

The only inaccuracy I found is with a very small start and large end, or vice versa:

julia> let start = 1, end_ = 1e15, len = 100, incr = (end_ - start) / (len - 1);
       (start + incr * (len-1)) - end_
       end
0.125

julia> let start = 1e15, end_ = 1, len = 100, incr = (end_ - start) / (len - 1);
       (start + incr * (len-1)) - end_
       end
-0.125

The error is proportional to the smaller endpoint. For this particular error to crop up, all of the other elements will be much larger than the smaller endpoint, so the relative error is tiny.

[c42f]

Hmm, looks like this problem has quite some history already: #2333 #9637 and probably several other issues.

I expect the aim of linspace should be to produce something as close as possible to the correctly rounded linear interpolation between start and end, hitting the end points exactly. Perhaps that's just me though.

The single sided version does have a nasty failure case when start is something large and negative and stop is small. In this case the relative errors on the right hand side of the range can be huge:

julia> x1,xN,N = -1f6, 0.1f0, 1e6
(-1.0f6,0.1f0,1.0e6)

julia> function lerp_onesided{T}(x1::T, xN::T, N, i)
           mult = (xN-x1)/(N-1)
           x1 + mult*(i-1)
       end
lerp_onesided (generic function with 1 method)

julia> (lerp_onesided(x1,xN,N,N) - xN)/xN
0.2499999802093956

julia> (linspace(x1,xN,N)[Int(N)] - xN)/xN
0.0f0

[c42f]

With a simple use of nextfloat and prevfloat during the LinSpace setup stage it seems possible to

• Exactly hit the end points
• Compute the values at each index without division
• Have small error relative to the correctly rounded result across the entire range when both endpoints share the same sign. (I suspect that getting this for ranges which cross zero isn't possible without extra logic.)

function lerp_fix{T}(x1::T, xN::T, N, i)
    # Setup.  Compute values at end points.
    mult = one(T)/T(N-1)
    a    = T(N-1)*mult*x1
    b    = T(N-1)*mult*xN
    # If they don't match, adjust so that linear interpolation formula exactly
    # hits the ends of the range.
    if a != x1
        x1 = a < x1 ? nextfloat(x1) : prevfloat(x1)
    end
    if b != xN
        xN = b < xN ? nextfloat(xN) : prevfloat(xN)
    end

    # The actual computation at index `i`
    T(N-i)*mult*x1 + T(i-1)*mult*xN
end

I haven't had time to test the above as extensively as I'd like yet, but it seemed to work when I tried it for a large number of log and uniformly distributed end points and range sizes.

[ringw]

That seems good. I can't find a proof, but it does seem that the error in "x*(1.0 / x)" is at most 1 in the mantissa, so either "nextfloat" or "prevfloat" should be able to fix the endpoints. If there's not a strong argument that this always works, it might be good to check that the updated endpoints are correct if a != x1 || b != xN, and raise an exception otherwise.

[c42f]

I think a proof of this is required if it's going to go into julia Base. So far I've come to a similar conclusion - first prove that

(1/M)*M ∈ {prevfloat(1),1,nextfloat(1)}

Follow that by proving that adjusting the endpoints by 1ulp is enough to correct a possible 1ulp error in (1/M)*M.

I haven't tried anything like this before, so I'm making slow progress. (Ok, I should probably just read the introduction in a good book on numerical analysis and it might be obvious. Suggestions welcome.)

Some observations which might be useful:

• M = len-1 is an integer, and exactly representable given the checking already in linspace().
• Floating point multiplication is exact if the input has few enough significant binary digits.

As a side note, lerp_fix fails if the rounding mode has been set to something other than RoundingMode{:Nearest}. I'm not too worried about this (taking the view that non-nearest rounding modes only make sense when local and lexically scoped), but perhaps other people disagree?

[StefanKarpinski]

I think there's a way to fix this and retain the LinSpace type, but there's been so much gnashing of teeth over this that I'm very strongly inclined to just revert to having linspace return a vector.

[andreasnoack]

@StefanKarpinski Isn't the Vector vs LinSpace discussion separate from the discussion about which values to return? Is it because two divisions per getindex would be too slow?

[StefanKarpinski]

It's related because the LinSpace construction makes it much harder to get the right behavior for the endpoints. I'm fairly certain that the (len-1)/divisor * start + (i-1)/divisor * end_ formula does not work for many of the nastier cases – I tried that out when I was implementing this. In order to get all the intermediate points right, you really need to do the addition at a "higher level" and then use a single division to project down and let the guaranteed correct rounding of the fdiv operation do its thing. If you try to do the addition after that, it defeats the correctness of the rounding.

[StefanKarpinski]

Btw, the potential solution that I have in mind for this that doesn't require ditching LinSpace is to multiply the numerator and divisor of the LinSpace object by a small multiplier until you get a starting point for which the projection is correct. I believe this always works, but I haven't been able to prove it. Still, it seems too complicated and makes me think that the LinSpace business should just be partially reverted to the point where we had lifted computations but returned an array.