Skip to content

Conversation

@SigureMo
Copy link
Member

@SigureMo SigureMo commented Mar 22, 2024

PR types

Bug fixes

PR changes

Others

Description

修复 AST 动转静 IfElseTransformer 转写后将含类型注释的变量变为 nonlocal 变量而导致 SyntaxError 的问题,问题源自 PaddlePaddle/PaddleScience#815 (comment)

cc @HydrogenSulfate

比如

paddle-py310TRANSLATOR_VERBOSITY=1 TRANSLATOR_CODE_LEVEL=16 python test.py
Fri Mar 22 06:32:15 Dynamic-to-Static INFO: (Level 1) Source code: 
@paddle.jit.to_static(full_graph=True)
def foo(x: paddle.Tensor):
    if x:
        y: List['paddle.Tensor'] = x + 1
    else:
        y: List['paddle.Tensor'] = x - 1
    return y
Fri Mar 22 06:32:15 Dynamic-to-Static INFO: After the level 16 ast transformer: 'TypeHintTransformer', the transformed code:
def foo(x):
    y = _jst.UndefinedVar('y')
    __return_0 = False
    __return_value_0 = None

    def get_args_0():
        nonlocal y
        return (_jst.Ld(y),)

    def set_args_0(__args):
        nonlocal y
        (y,) = _jst.Ld(__args)

    def true_fn_0():
        nonlocal y
        y: _jst.Ld(_jst.Ld(List)['paddle.Tensor']) = _jst.Ld(x) + 1
        return

    def false_fn_0():
        nonlocal y
        y: _jst.Ld(_jst.Ld(List)['paddle.Tensor']) = _jst.Ld(x) - 1
        return
    _jst.IfElse(_jst.Ld(x), _jst.Ld(true_fn_0), _jst.Ld(false_fn_0), _jst.Ld(get_args_0), _jst.Ld(set_args_0), ('y',), push_pop_names=None)
    __return_value_0 = _jst.Ld(y)
    return _jst.Ld(__return_value_0)
Fri Mar 22 06:32:15 Dynamic-to-Static WARNING: Please file an issue at 'https://github.com/PaddlePaddle/Paddle/issues' if you can't handle this <class 'SyntaxError'> yourself.
Traceback (most recent call last):
  File "/workspace/Paddle/test.py", line 39, in <module>
    foo(paddle.to_tensor(1))
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 502, in __call__
    return self._perform_call(*args, **kwargs)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 825, in _perform_call
    raise e
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 794, in _perform_call
    _, partial_program_layer = self.get_concrete_program(
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 876, in get_concrete_program
    concrete_program, partial_program_layer = self._program_cache[
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 1685, in __getitem__
    self._caches[item_id] = self._build_once(item)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 1610, in _build_once
    concrete_program = ConcreteProgram.from_func_spec(
  File "/home/nyakku/mambaforge/envs/paddle-py310/lib/python3.10/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/workspace/Paddle/build/python/paddle/base/wrapped_decorator.py", line 26, in __impl__
    return wrapped_func(*args, **kwargs)
  File "/workspace/Paddle/build/python/paddle/base/dygraph/base.py", line 66, in __impl__
    return func(*args, **kwargs)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 1292, in from_func_spec
    static_func = convert_to_static(dygraph_function)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 188, in convert_to_static
    static_func = _FUNCTION_CACHE.convert_with_cache(function)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 113, in convert_with_cache
    static_func = self._convert(func)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/program_translator.py", line 153, in _convert
    static_func, file_name = ast_to_func(root, func)
  File "/workspace/Paddle/build/python/paddle/jit/dy2static/utils.py", line 320, in ast_to_func
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/nyakku/.cache/paddle/to_static_tmp/27002/foom17kuuup.py", line 24
    y: _jst.Ld(_jst.Ld(List)['paddle.Tensor']) = _jst.Ld(x) + 1
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: annotated name 'y' can't be nonlocal

原来 y 转写后变成了 nonlocal 变量,而 nonlocal 变量是不允许加类型注释的

因此将 TypeHintTransformer 从最后一个移到最前面,并添加了 AnnAssign 的处理(同样是删除 annotation,变成普通的 Assgin node)

PCard-66972

@paddle-bot
Copy link

paddle-bot bot commented Mar 22, 2024

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@SigureMo SigureMo requested a review from gouzil March 22, 2024 06:57
@SigureMo SigureMo changed the title [Dy2St] Move TypeHintTransformer before IfElseTransformer [Dy2St] Move TypeHintTransformer ahead of IfElseTransformer Mar 22, 2024
gouzil
gouzil previously approved these changes Mar 22, 2024
@SigureMo SigureMo merged commit 4836971 into PaddlePaddle:develop Mar 25, 2024
@SigureMo SigureMo deleted the dy2st/move-type-hint-transformer-before-if-else-transformer branch March 25, 2024 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants