Add squared_l2_distance_op#3768
Conversation
pkuyym
changed the title
| 'X': np.random.uniform(0.1, 1., (2, 3)).astype('float32'), | ||
| 'Y': np.random.uniform(0.1, 1., (2, 3)).astype('float32') | ||
| } | ||
| self.check_grad(op, inputs, set(["X", "Y"]), "Out") |
There was a problem hiding this comment.
Add compare_grad to check the consistency of results for GPU and CPU calculation.
|
I think this op is well written. @pkuyym has enhanced functionality compared with the raw implementation of Paddle. And this op also has a good error reporting information. |
| auto* x_grad = ctx.Output<Tensor>(framework::GradVarName("X")); | ||
| auto* y_grad = ctx.Output<Tensor>(framework::GradVarName("Y")); | ||
| if (x_grad != nullptr) x_grad->Resize(x_dims); | ||
| if (y_grad != nullptr) y_grad->Resize(y_dims); |
There was a problem hiding this comment.
if (x_grad) ...
if (y_grad) ...|
|
||
| // propagate back to input | ||
| auto eigen_place = context.GetEigenDevice<Place>(); | ||
| if (x_g != nullptr) { |
| "inputs must be same."); | ||
|
|
||
| int rank = framework::arity(x_dims); | ||
| PADDLE_ENFORCE(rank >= 2, "Tensor rank should be at least equal to 2."); |
There was a problem hiding this comment.
PADDLE_ENFORCE_GE(rank, 2, "doc")
just like GLOG
| framework::product(y_dims) / y_dims[0], | ||
| "Product of dimensions expcet the first dimension of " | ||
| "input and target must be equal."); | ||
| PADDLE_ENFORCE(y_dims[0] == 1 || y_dims[0] == x_dims[0], |
There was a problem hiding this comment.
split this enforce to two paddle_enforces or print whether y_dims[0]==1 or y_dims[0]==x_dims[0] make this enforce fail.
make this condition check more concise.
There was a problem hiding this comment.
y_dims[0] == 1 || y_dims[0] == x_dims[0] is an atomic assertion. I think it's difficult to split to two assertions.
There was a problem hiding this comment.
@pkuyym that's right.
There is a typo expcet need to fix, you can use the grammarly to correct them before pushing to github.
| auto y_grad = | ||
| EigenMatrix<T>::From(*y_g, framework::make_ddim({y_dims[0], cols})); | ||
|
|
||
| PADDLE_ENFORCE(sub_result.dimensions()[0] >= y_dims[0], |
| void InferShape(const framework::InferShapeContext& ctx) const override { | ||
| PADDLE_ENFORCE_NOT_NULL(ctx.InputVar(framework::GradVarName("Out")), | ||
| "Gradient of Out should not be null"); | ||
| // check out grad dimensions |
There was a problem hiding this comment.
maybe this line of comment can be removed?
| out0->mutable_data<T>(context.GetPlace()); | ||
| out1->mutable_data<T>(context.GetPlace()); | ||
| auto sub_result = EigenMatrix<T>::From(*out0); | ||
| auto z = EigenMatrix<T>::From(*out1); |
| } | ||
| auto sub_res_pow2 = sub_result * sub_result; | ||
| // z is TensorMap, no need reshape | ||
| z.device(place) = sub_res_pow2.sum(Eigen::array<int, 1>({1})); |
There was a problem hiding this comment.
Please change to and refer to https://stackoverflow.com/questions/31555584/why-is-clang-warning-suggest-braces-around-initialization-of-subobject-wmis
z.device(place) = sub_res_pow2.sum(Eigen::array<int, 1>({{1}}));
| int cols = framework::product(x_dims) / x_dims[0]; | ||
| // calculate gradient | ||
| auto grad_mat = | ||
| 2 * (out_grad.broadcast(Eigen::array<int, 2>({1, cols}))) * sub_result; |
| if (y_g) { | ||
| y_g->mutable_data<T>(context.GetPlace()); | ||
| auto y_grad = | ||
| EigenMatrix<T>::From(*y_g, framework::make_ddim({y_dims[0], cols})); |
There was a problem hiding this comment.
Please think about the rank of y_grad carefully. The result of Eigen::Sum() will reduce dimension
5 participants
resolves #3736