Add additional dimensions support in Dense layer.

[harryfei]

By supporting additional dimensions, the interface of Dense layer would be like this.

• Input Shape: (in_features, d1, d2, d3, ..., N)
• Ouput Shape: (out_features, d2, d2, d3,..., N)

Related discuss: #282

[MikeInnes]

Thanks! A couple comment before we can merge this –

I think it'd be cleaner to keep the current Dense definition as-is, perhaps with a AbstractVecOrMat type restriction. Then you can overload AbstractArray and do the needed reshaping.

You can do this in a single matmul by just reshaping the inputs/outputs to Dense appropriately, which would avoid extra copies and be more GPU-friendly.

[harryfei]

I have no idea in my mind, could you explain more?

Update: Never mind, I get it.

[harryfei]

I think we don't need this change anymore according to the above cleaner solution.
Should we add the reshape helper function in this repository? If it's Ok, I will create another PR. @MikeInnes

[MikeInnes]

Sure, if you can update this PR with the reshape version, that works great for me.

[drozzy]

Any eta on when this will be merged? I'm finding myself missing this basic functionality.
I wish all layers treated extra dimensions as batch dimensions.

[drozzy]

Tested and it works for me:

a = randn(10, 11, 12) 
b =  Dense(10, 20)
b(a) # 20x11x12

and more!

a = randn(10, 11, 15, 12) 
b =  Dense(10, 20)
b(a) # 20×11×15×12

[CarloLucibello]

this is not an efficient way to do it, we should go through a reshape

[drozzy]

Yes, also I tried copying this code to my implementation and I seem to be having some issues with gpu. Don't quote me on this.

I would be happy for reshape to be merged, as long as we get something to help out with the case I provided above.
Thanks.