Views of Flux OneHotArrays

[RS-Coop]

The behviour of views and related functionality on CUDA arrays behaves as expected, but causes problems on Flux OneHotArrays. Specifically, the indices of the views aren't stored in a CUDA array which causes scalar indexing issues and an isbits error. The following example gives some insight into the issue, although it doesn't show the errors that eventually crop up:

using Flux, CUDA

data = rand(1:10, 10, 4)

x = data |> gpu
xhot = Flux.onehotbatch(data, 1:10) |> gpu

typeof(view(x, :, [1,3]))
#=
SubArray{Int64, 2, CuArray{Int64, 2, CUDA.Mem.DeviceBuffer},
Tuple{Base.Slice{Base.OneTo{Int64}}, CuArray{Int64, 1, CUDA.Mem.DeviceBuffer}}, false}
=#

typeof(view(xhot, :, :, [1,3]))
#=
SubArray{Bool, 3, Flux.OneHotArray{UInt32, 10, 2, 3, CuArray{UInt32, 2, CUDA.Mem.DeviceBuffer}},
Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, Vector{Int64}}, false}
=#

This may be too niche of a problem, but it seems possible this could be an issue in other places where the structure isn't a CUDA array itself, but uses CUDA arrays. It would be nice if this could be automatically detected, and then view would put all indices on CUDA arrays.

The following code gets around the issue, but it doesn't seem ideal:

xv = view(x, :, [1,3])
typeof(Base.unsafe_view(xhot, Base.Slice(Base.OneTo(10)), Base.Slice(Base.OneTo(10)), xv.indices[ndims(x)]))

[maleadt]

Specifically, the indices of the views aren't stored in a CUDA array which causes scalar indexing issues and an isbits error.

CuArray is only used to represent contiguous or strided arrays, which isn't the case when you have a view with indices. The fact that some views are represented by a CuArray is already a stretch -- only done so because of load time issues when using SubArray{..., <:CuArray} everywhere -- and I don't think we should extend this to views with indices as these cannot be passed to e.g. CUBLAS or other NVIDIA libraries.

The fact that SubArray{..., Flux.OneHotArray{<:CuArray}} isn't a CuArray and doesn't have the specialized methods CUDA.jl provides for a singly-wrapped SubArray{..., <:CuArray} is something that should be fixed in Base, e.g. with JuliaLang/julia#31563 which would allow to identify the wrapped CuArray without a combinatorial explosion.

[RS-Coop]

Just so I have the correct understanding, can you answer the following two questions:

• Is the idea of views of CuArrays fundamentally impractical because they are supposed to be contiguous?
• Is it possible to use sparse CuArrays instead of Flux one hot arrays to achieve the same goal?

Thanks for the info

[maleadt]

Is the idea of views of CuArrays fundamentally impractical because they are supposed to be contiguous?

No, views are just supported. Contiguous ones are represented by CuArray objects, all other views just use SubArray. Nothing impractical here.

Is it possible to use sparse CuArrays instead of Flux one hot arrays to achieve the same goal?

That depends on Flux, but I think there's going to be missing functionality. E.g. CuSparseArray doesn't support broadcast.

[RS-Coop]

Okay, thanks for taking the time to answer my questions!