Skip to content

Handling the three-way exits of generators and coroutines efficiently. #448

@markshannon

Description

@markshannon

Python has two kinds of callables: functions and generators/coroutines.

Functions have a two-way exit: return or raise.
Coroutines have a three-way exit: return, yield or raise.

Because generators and coroutines are newer, a lot of the C code in genobject.c, ceval.c and related files squeezes the three-way exit into a two-way exit using StopIteration and StopAsyncIteration which is really clunky and inefficient.

Instead of squeezing three-way exits into two-ways exits, we should be broadening the two-way exits into three-way exits, when needed.

For example, we can drop the throwflag argument from _PyEval_EvalFrameDefault by implementing gen.throw() in bytecode.
We can do this because bytecode already handles the three-way exit in FOR_ITER and SEND as follows:

  • Return: jump
  • Yield: push value to stack
  • Raise: propagate.

gen.send() can be implemented as something like:

  LOAD_FAST     0 (self)
  LOAD_FAST     1 (value)
  SETUP_FINALLY exception_handler
  SEND               returned
  POP_BLOCK
  RETURN_VALUE
returned:
  LOAD_CONST       StopIteration
  PUSH_NULL
  SWAP 3
  CALL                     1
  RAISE_VARARGS            1
exception_handler:
  LOAD_CONST       StopIteration
  CHECK_EXC_MATCH
  POP_JUMP_FORWARD_IF_FALSE    reraise
  POP_TOP
  PUSH_NULL
  LOAD_CONST      RuntimeError
  CALL                     0
  RAISE_VARARGS            1
reraise:
   RERAISE                  0
     

gen.throw(), etc. can be implemented similarly.
See #67 (comment)

Of course, we still need to have some C functions, not everything can be done in bytecode.
For those functions, we should use something like the interface of gen_send_ex2 which returns the kind of exit, and uses an out parameter for the value

PySendResult gen_send_ex2(..., PyObject **presult ,..);

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions