add finite_difference_jvp

[oscardssmith]

Add the pushforward operation with implementation taken from jacobian but simplified. This is my first time touching this code, so let me know if there's anything obviously wrong here.

[gdalle]

Thank you for giving it a try! I'll try to review a bit later but I think like the other operators, this would be most useful with a Cache?

[ChrisRackauckas]

Yes it needs a cache.

[gdalle]

Hi @oscardssmith, friendly ping on this one, do you think you can complete it? Otherwise the performance of sparse Jacobians with DI+FiniteDiff will be bad, so I think this might be a priority before switching OrdinaryDiffEq to DI.

[oscardssmith]

oh yeah, I can finish this off.

[oscardssmith]

ok, this is finally updated to have a VJP cache. I've given it a little testing, and will add some actual tests, but I think this is mostly ready to merge!

[gdalle]

Thanks @oscardssmith! I don't see any mention of a Cache type in the PR changes though, are you sure we mean the same thing by it?

[oscardssmith]

something went very wrong in the git process

[gdalle]

[oscardssmith]

now it actually contains the thing it was supposed to :)

[gdalle]

This version passes my preliminary DI correctness tests! Note that I didn't benchmark performance or allocations or anything, I just used your code inside DI.pushforward(f, ::AutoFiniteDiff, ...).

For testing here, perhaps the easiest option is to augment the jacobian tests with their JVP counterparts?

[gdalle]

Cross-referencing what will become the ensuing DI PR: JuliaDiff/DifferentiationInterface.jl#705

[oscardssmith]

Tests added (and some of the forward mode tests for jacobians have been strengthened from 1e-4 to 1e-6 because they were aggressively loose tests before)

[oscardssmith]

Tests are passing. Time to merge?

[gdalle]

I can make a review pass tomorrow but I'm not familiar with this package so I'd rather have someone more confident with FiniteDiff give an opinion?

[oscardssmith]

Looks like @ChrisRackauckas has merged all PRs in the past couple months so do you want to review?

[gdalle]

well it does make a copy of x

[ChrisRackauckas]

yes the non-allocating cache is expected to just use the arrays as given 😅

[gdalle]

we should probably also increment the minor version?

[gdalle]

Any way to get code coverage information? Codecov protests here (and on master too):

Upload failed: {"message":"Token required because branch is protected"}

[gdalle]

gentle ping, this is basically finished I think, although codecov would be nice?