Make for iter::CartesianIndices better vectorized for 1d/2d cases.

[N5N3]

Local benchmark shows that this makes non-@simd 1d/2d loop faster.

function sumcart_iter(A)
    s = zero(eltype(A))
    for I in CartesianIndices(A)
        @inbounds @fastmath s += A[I] # use @fastmath to enable simd
    end
    s
end

on master:

julia> A = view(rand(256*20),1:256*20);
julia> @btime sumcart_iter($A)
  4.657 μs (0 allocations: 0 bytes)
2539.6790676561955

julia> B = view(rand(256,20),1:256,1:20);
julia> @btime sumcart_iter($B)
  4.657 μs (0 allocations: 0 bytes)
2557.7341880811764

This PR

julia> @btime sumcart_iter($A)
  305.200 ns (0 allocations: 0 bytes)
2539.6790676562046

julia> @btime sumcart_iter($B)
  406.500 ns (0 allocations: 0 bytes)
2557.734188081178

[johnnychen94]

I'm not sure if I get the idea -- when will N != 1 here?

[N5N3]

N is the dimension of the CartesianIndices:

1. If N > 1, the outermost dimension uses __is_valid_range to preserve the performance improvement introduced in add StepRange support for CartesianIndices #37829.
2. If N == 1, just use state[1] != last(rng), as __is_valid_range pervents vectorization.

[johnnychen94]

But by calling __inc(state.I, iter.indices, Val(ndims(iter))) as in you did in R404, because of the type annotion state::Tuple{Int}, indices::Tuple{OrdinalRangeInt}, this method R420 would only be called when length(iter.indices) == ndims(iter) == 1, right?

[N5N3]

It's also called at R435 as the last input ndim is passed deeper without any change.

BTW, all the test failures should be invalid state. Since we have

julia> iterate(1:2:typemax(Int), typemax(Int)-1)
(-9223372036854775808, -9223372036854775808)

I think it's OK to replace them.

[johnnychen94]

This is good for me as long as the test passes. Also ping @vchuravy and @timholy as this was originally written in #31011

[N5N3]

Some local BaseBenchmark:

          "index" => 7-element BenchmarkTools.BenchmarkGroup:
                  tags: ["sum", "simd"]
                  ("sumeach", "SubArray{Int32, 2, Matrix{Int32}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}") => TrialJudgement(-85.57% => improvement)
                  ("sumcartesian", "SubArray{Int32, 2, Matrix{Int32}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}") => TrialJudgement(-87.51% => improvement)
                  ("sumcartesian", "1:100000") => TrialJudgement(-100.00% => improvement)
                  ("sumeach", "SubArray{Int32, 2, BaseBenchmarks.ArrayBenchmarks.ArrayLS{Int32, 2}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}}, false}") => TrialJudgement(-87.62% => improvement)
                  ("sumcartesian", "SubArray{Int32, 2, BaseBenchmarks.ArrayBenchmarks.ArrayLS{Int32, 2}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}}, false}") => TrialJudgement(-87.48% => improvement)
                  ("sumcartesian", "SubArray{Int32, 2, Matrix{Int32}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}}, true}") => TrialJudgement(-84.04% => improvement)
                  ("sumcartesian", "100000:-1:1") => TrialJudgement(-100.00% => improvement)

[N5N3]

With today's master (LLVM14), this PR also helps vectorizing 3d CartesianIndices (Very limited though)

[N5N3]

No need after #51606.

[jonas-schulze]

It's a bit sad that (subjectively) so many PRs get forgotten about for so long.