-
Notifications
You must be signed in to change notification settings - Fork 5.9k
fix RNN and IfElse syntax in Block design #4210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
c7cd48c
59f3f3c
d52dffd
c5bd0f3
6a2bf6c
0b83e80
5c2ea17
bf7d99d
a3e1fd5
ead84fe
8b5e497
dae0007
c0bac84
f897535
2e844a1
50aeec4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -57,14 +57,17 @@ The following C++ programs shows how blocks are used with the `if-else` structur | |
| ```c++ | ||
| int x = 10; | ||
| int y = 20; | ||
| int out; | ||
| int z = 30; | ||
| int out, out1; | ||
| bool cond = false; | ||
| if (cond) { | ||
| int z = x + y; | ||
| out = softmax(z); | ||
| } else { | ||
| int z = fc(x); | ||
| out = z; | ||
| out1 = softmax(z); | ||
| } else { | ||
| int d = fc(z); | ||
| out = d; | ||
| out1 = d+1; | ||
| } | ||
| ``` | ||
|
|
||
|
|
@@ -73,48 +76,59 @@ An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator | |
| ```python | ||
| import paddle as pd | ||
|
|
||
| x = var(10) | ||
| y = var(20) | ||
| x = var([10, 20]) | ||
| # a scalar | ||
| y = var([1]) | ||
| z = var(10, 20) | ||
| cond = var(false) | ||
| ie = pd.create_ifelseop(inputs=[x], output_num=1) | ||
| # output_num should be set to ensure the outputs of the true_block and false_block can be merged, | ||
| # so the numbers of both blocks should be same as `output_num`. | ||
| ie = pd.ifelse_builder(output_num=2) | ||
|
|
||
| with ie.true_block(): | ||
| x = ie.inputs(true, 0) | ||
| z = operator.add(x, y) | ||
| ie.set_output(true, 0, operator.softmax(z)) | ||
| x_ = x.as_ifelse_input() | ||
|
||
| z = operator.add(x_, y) | ||
| ie.set_outputs(z, operator.softmax(z)) | ||
|
|
||
| with ie.false_block(): | ||
| x = ie.inputs(false, 0) | ||
| z = layer.fc(x) | ||
| ie.set_output(true, 0, operator.softmax(z)) | ||
| z_ = z.as_ifelse_input() | ||
| d = layer.fc(z_) | ||
| ie.set_outputs(d+1, operator.softmax(d)) | ||
|
|
||
| out = b(cond) | ||
| ``` | ||
|
|
||
| In both examples, the left branch computes `softmax(x+y)` and the right branch computes `fc(x)`. | ||
| In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `x+1` and `fc(x)`. | ||
|
|
||
| A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values. | ||
|
|
||
|
|
||
| ### Blocks with `for` and `RNNOp` | ||
|
|
||
| The following RNN model from the [RNN design doc](./rnn.md) | ||
|
|
||
| ```python | ||
| x = sequence([10, 20, 30]) | ||
| # x is a LoDTensor and stores sequences | ||
| x = var(sequence=([10, 20, 30])) | ||
| m = var(0) | ||
| W = tensor() | ||
| U = tensor() | ||
|
|
||
| rnn = create_rnn(inputs=[input]) | ||
| with rnn.stepnet() as net: | ||
| x = net.set_inputs(0) | ||
| h = net.add_memory(init=m) | ||
| fc_out = pd.matmul(W, x) | ||
| hidden_out = pd.matmul(U, h.pre(n=1)) | ||
| W = var() | ||
| U = var() | ||
|
|
||
| rnn = rnn_builder() | ||
| with rnn.stepnet(): | ||
| # mark the variables that need to be segmented for time steps. | ||
| x_ = x.as_step_input() | ||
| # mark the varialbe that used as a RNN state. | ||
| h_ = h.as_step_memory(init=m) | ||
|
||
|
|
||
| fc_out = pd.matmul(W, x_) | ||
| hidden_out = pd.matmul(U, h_.pre(nstep=1)) | ||
| sum = pd.add_two(fc_out, hidden_out) | ||
| act = pd.sigmoid(sum) | ||
| h.update(act) # update memory with act | ||
| net.set_outputs(0, act, hidden_out) # two outputs | ||
| h_.update(act) # update memory with act | ||
| net.set_outputs(act, hidden_out) # two outputs | ||
|
|
||
| o1, o2 = rnn() | ||
| print o1, o2 | ||
| ``` | ||
|
|
||
| has its equivalent C++ program as follows | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a replacement here #4313