Skip to content
Closed
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions doc/design/python_interface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Python Interface Design

## Variable as variable data type (not parameter)
The class Variable is introduced to define the variables, which are not only parameters but also the input data and intermediate outputs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The class Variable is ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get


All the variables are defined in a variable scope, `Scope`, it will check duplicate variable names.

The `Variable`'s definition is as follows
```python
class UniqueNameGenerator:
counter = 0

def __call__(self, prefix):
name = '%s-%d' % (prefix, counter)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe %s.%d is better. Because %s-%d is not a valid filename on some platforms, e.g., Windows.

counter += 1
return name


unique_name_generator = UniqueNameGenerator()


class Variable(object):
def __init__(self, shape, name=None, prefix=None, var_scope=None, op=None):
'''
var_scope is VarDesc's scope.
'''
if name is None:
if prefix is not None:
name = unique_name_generator(prefix)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this design. We don't need to call unique_name_generator in layer definitions.

else:
name = unique_name_generator("unknown")
self.name = name
self.var_scope = var_scope
self.op = op
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to define a varibale's op? If two operators write the same variable, which one is its op?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the last one who write it? @reyoung @jacquesqiao


def shape(self):
pass
```

in above example, the `unique_name_generator` is introduced to make unique name with a prefix and can be used to generate variables' and operators' names.

## Scope Stack for Block Inherience
Each block should has a variable scope that is inheriented from parent's variable scope.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should have. 情态动词后不用第三人称单数形式。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get

That needs a stack of variable scopes, so that when user's model definition jumps out a sub-block,
the following variables could be defined in current variable scope.

For example, when using a RNNOp

```python
# v is output of some op
v = someop()

rnn = RNNOp()
with rnn.stepnet():
# current in a sub-block
# all the variables are stored in a sub variable scope
c = pd.fc(v)

e = someop()
```

first, the `v` variable is defined in global variable scope,
while `c` is in rnn's sub-variable scope,
the `e` variable is defined outside the rnn's stepnet, and is registered in global scope again.

Obviously, a variable scope is needed to help switch variale scopes.

There is another important issue in above code snippet, the `pd.fc` used inside rnn's stepnet will create parameters in some variable scope,
currently, the trainable parameters should be registered in global scope so that the gradient optimizers can find and update them.

So, the parameters should only created in the head of the variable stack, we can implement this stack like

```python
class ScopeStack(object):
def __init__(self):
self.scope = [Scope()]

def push_stack(self):
self.scope.push(Scope())

def pop_stack(self):
self.scope.pop()

def cur_scope(self):
return self.scope[-1]

def global_scope(self):
return self.scope[0]


g_scope_stack = ScopeStack()


# `new_variable` could be based on the `g_scope_stack`, and could be based on a user-defined scope stack.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users are able to build their own scope stack? What's that for?

def new_variable(scope_stack=None):
if scope_stack is None:
scope_stack = g_scope_stack
return scope_stack.cur_scope.new()
```