Currently, our operators use the parameter name of input/output to get the input/output variable names. For example, we are using Input("X"), Input("W"), and Input("B") in fully connected operator to get the data, weight matrix and bias. But using string is easy to get wrong, our framework should check whether that parameter name is registered in OpInfo or not. The parameter name is saved in OpProto.Var.name.
We can do this by searching the parameter name in OpProto directly. However, this implementation has several issues:
- The time complexity for validating a parameter name directly by
OpProto is O(n), which n is the number of inputs/outputs since it is stored in a linear list.
- Gradient operators do not have
OpProto.
The simpler way is to add input_params and output_params sets to OpInfo as follow:
struct OpInfo {
set<string> input_params;
set<string> output_params;
};The OpInfo should be constructed while registering operator. It works well for both forwarding and backward operators.
Currently, our operators use the parameter name of input/output to get the input/output variable names. For example, we are using
Input("X"),Input("W"), andInput("B")in fully connected operator to get the data, weight matrix and bias. But using string is easy to get wrong, our framework should check whether that parameter name is registered inOpInfoor not. The parameter name is saved inOpProto.Var.name.We can do this by searching the parameter name in
OpProtodirectly. However, this implementation has several issues:OpProtois O(n), which n is the number of inputs/outputs since it is stored in a linear list.OpProto.The simpler way is to add
input_paramsandoutput_paramssets toOpInfoas follow:The
OpInfoshould be constructed while registering operator. It works well for both forwarding and backward operators.