11"""
22 AbstractZero <: AbstractDifferential
33
4- This is zero-like differential types.
5- If a AD system encounter a propagator taking as input only subtypes of `AbstractZero` then
6- it can stop performing any AD operations, as all propagator are linear functions, and thus
7- the final result will be zero.
4+ Supertype for zero-like differentials—i.e., differentials that act like zero when
5+ added or multiplied to other values.
6+ If an AD system encounters a propagator that takes as input only subtypes of `AbstractZero`,
7+ then it can stop performing AD operations.
8+ All propagators are linear functions, and thus the final result will be zero.
89
910All `AbstractZero` subtypes are singleton types.
10- There are two of them [`Zero()`](@ref) and [`DoesNotExist()`](@ref).
11+ There are two of them: [`Zero()`](@ref) and [`DoesNotExist()`](@ref).
1112"""
1213abstract type AbstractZero <: AbstractDifferential end
1314Base. iszero (:: AbstractZero ) = true
@@ -28,7 +29,7 @@ Base.:/(z::AbstractZero, ::Any) = z
2829
2930The additive identity for differentials.
3031This is basically the same as `0`.
31- A derivative of `Zero()`. does not propagate through the primal function.
32+ A derivative of `Zero()` does not propagate through the primal function.
3233"""
3334struct Zero <: AbstractZero end
3435
@@ -43,21 +44,22 @@ Base.zero(::Type{<:AbstractDifferential}) = Zero()
4344 DoesNotExist() <: AbstractZero
4445
4546This differential indicates that the derivative does not exist.
46- It is the differential for a Primal type that is not differentiable.
47- Such an Integer, or Boolean (when not being used as a represention of a value that normally
48- would be a floating point.)
49- The only valid way to pertube such a values is to not change it at all.
50- As such , `DoesNotExist` is functionally identical to `Zero()`,
51- but provides additional semantic information.
47+ It is the differential for primal types that are not differentiable,
48+ such as integers or booleans (when they are not being used to represent
49+ floating- point values).
50+ The only valid way to perturb such values is to not change them at all.
51+ As a consequence , `DoesNotExist` is functionally identical to `Zero()`,
52+ but it provides additional semantic information.
5253
53- If you are adding this differential to a primal then something is wrong.
54- A optimization package making use of this might like to check for such a case.
54+ Adding this differential to a primal is generally wrong: gradient-based
55+ methods cannot be used to optimize over discrete variables.
56+ An optimization package making use of this might want to check for such a case.
5557
5658!!! note:
57- This does not indicate that the derivative it is not implemented,
59+ This does not indicate that the derivative is not implemented,
5860 but rather that mathematically it is not defined.
5961
60- This mostly shows up as the deriviative with respect to dimension, index, or size
62+ This mostly shows up as the derivative with respect to dimension, index, or size
6163arguments.
6264```
6365 function rrule(fill, x, len::Int)
0 commit comments