Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions doc/design/python_interface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Python Interface Design

## Variable as variable data type (not parameter)
The class Variable is introduced to define the variables that are not parameters.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The class Variable is introduced to define the variables, which are not only parameters but also the input data and intermediate outputs.


All the variables are defined in a variable scope, `VarScope`, it will check duplicate variable names.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that Scope is enough?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this design is based on current implementation ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. This design should handle current implementation and future compile time concepts.

However, is the compile time concept of Scope needs a different name? because in that time, the user cannot access runtime scope directly, so there is only one concept which related to scope in future?


```python
class VarScope(object):
'''
VarScope is names' container of VarDesc in compile period.
'''
pass
```

The `Variable`'s definition is as follows
```python
class _unique_name_generator:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP8, class name is upper case

class UniqueNameGenerator(object):
   ...

counter = 0

def __call__(self, prefix):
name = '%s-%d' % (prefix, counter)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe %s.%d is better. Because %s-%d is not a valid filename on some platforms, e.g., Windows.

counter += 1
return name


unique_name_generator = _unique_name_generator()

class Variable(object):
def __init__(self,
shape,
name=None,
prefix=None,
var_scope=None,
op=None,
reuse=False):
'''
var_scope is VarDesc's scope.
'''
self.shape = shape
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shape is not stored in Variable. It stored in tensor.

if name is None:
if prefix is not None:
name = unique_name_generator(prefix)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this design. We don't need to call unique_name_generator in layer definitions.

else:
name = unique_name_generator("unknown")
self.name = name
self.var_scope = var_scope
if not reuse:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may not be Variable's responsibility to check reuse.

assert self.name not in self.var_scope, "variable name %s duplicate in var scope" % self.name
self.op = op
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to define a varibale's op? If two operators write the same variable, which one is its op?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the last one who write it? @reyoung @jacquesqiao

```

in above example, the `unique_name_generator` is introduced to make unique name with a prefix and can be used to generate variables' and operators' names.

## VarScope Stack for Block Inherience
Each block should has a variable scope that is inheriented from parent's variable scope.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should have. 情态动词后不用第三人称单数形式。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get

That needs a stack of variable scopes, so that when user's model definition jumps out a sub-block,
the following variables could be defined in current variable scope.

For example, when using a RNNOp

```python
# v is output of some op
v = someop()

rnn = RNNOp()
with rnn.stepnet():
# current in a sub-block
# all the variables are stored in a sub variable scope
c = pd.fc(v)

e = someop()
```

first, the `v` variable is defined in global variable scope,
while `c` is in rnn's sub-variable scope,
the `e` variable is defined outside the rnn's stepnet, and is registered in global scope again.

Obviously, a variable scope is needed to help switch variale scopes.

There is another important issue in above code snippet, the `pd.fc` used inside rnn's stepnet will create parameters in some variable scope,
currently, the trainable parameters should be registered in global scope so that the gradient optimizers can find and update them.

So, the parameters should only created in the head of the variable stack, we can implement this stack like

```python
g_varscope_stack = []
Copy link
Collaborator

@reyoung reyoung Sep 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

global variables are not good, especially for multi-thread situation.

Maybe we can,

class ScopeStack(object):
  def __init__(self):
    self.scope = [Scope()]

  def push_stack(self):
    self.scope.push(Scope())

  def pop_stack(self):
    self.scope.pop()

  def cur_scope(self):
    return self.scope[-1]

  def global_scope(self):
    return self.scope[0]

g_scope_stack = ScopeStack()

# `new_variable` could be based on the `g_scope_stack`, and could be based on a user-defined scope stack.
def new_variable(scope_stack=None):
  if scope_stack is None:
    scope_stack = g_scope_stack
  return scope_stack.cur_scope.new()



class VarScope(object):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VarScope may not be exposed to users. It should be managed with sub-block or sub-net.

So, let us not write it in this design doc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some code snippets, not the actual definition. Explain something about a variable scope, no matter what its name is, Scope or VarScope.

Even the original Scope is used, it is better to use a python adapter may be called VarScope so that when we replace Scope easily in Variable latter.

def __enter__(self):
g_varscope_stack.append(self)

def __exit__(self, type, value, traceback):
g_varscope_stack.pop()

if not g_varscope_stack: g_varscope_stack.append(VarScope())

def Parameter(...):
g_varscope_stack[0].register(...)
```