bug: functions defined in `markdown-exec` context don't have a `__module__`

[femtomc]

When I define functions inside a markdown-exec block, the functions don't possess the __module__ attribute:

def f(x):
    return x + 1.0

print(f.__module__)

returns None

This, unfortunately, interacts poorly with systems that expect a __module__ attribute (beartype/beartype#381)

For instance, I was attempting to setup some documentation to show my users what a beartype error would look like, but ran into this issue where beartype expects the callable which it is typechecking to have a __module__ attribute (beartype/beartype#381).

Is it possible to configure markdown-exec to supply a dummy module? Or to use the session string as the dummy module?

Environment

Working with: markdown-exec==1.8.1

[pawamoy]

Hi @femtomc, thanks for the report 🙂

I ran some experiment and it's indeed possible to fix this, by adding a __name__ value to the execution global context 🙂 Just have to pick a value that makes sense now 😄

Would beartype be OK with __name__ or __module__ having a value like <code block: n1>?

[femtomc]

I'm not entirely sure! I don't know if it does any further introspection on the module -- perhaps the maintainer will comment on my issue there and we'll know better

[pawamoy]

OK thanks 🙂 I think I'll go with a _code_block_n1_ instead, at least that makes it more compliant with what other tools usually expect a module name to look like. The module will never exist so I don't think we have to wait for beartype's maintainers to answer, I'll release a fix 🙂

[femtomc]

Thank you so much!

[pawamoy]

Ah, it's causing other issues unfortunately 🤔 I'm getting this traceback on my own executed code blocks now:

Traceback (most recent call last):
  File "/media/data/dev/markdown-exec/src/markdown_exec/formatters/python.py", line 61, in _run_python
    exec(compiled, exec_globals)  # noqa: S102
  File "<code block: session insiders; n2>", line 31, in <module>
    class Project:
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 1019, in dataclass
    return wrap(cls)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 1011, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 861, in _process_class
    cls_fields = [_get_field(cls, name, type)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 861, in <listcomp>
    cls_fields = [_get_field(cls, name, type)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 712, in _get_field
    and _is_type(f.type, cls, typing, typing.ClassVar,
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.8.18/lib/python3.8/dataclasses.py", line 658, in _is_type
    ns = sys.modules.get(cls.__module__).__dict__
AttributeError: 'NoneType' object has no attribute '__dict__'

[femtomc]

@pawamoy Not a big deal if can't fix on this side, I can do a bit of hacking (and see what the beartype side can do) to satisfy my needs

[pawamoy]

Seems I can fix it with this:

    module_name = re.sub(r"[^a-zA-Z\d]+", "_", code_block_id)
    exec_globals["__name__"] = module_name
    sys.modules[module_name] = ModuleType(module_name)

I worry about other potential side-effects 🤔

[pawamoy]

I'll push it, and if it breaks I'll revert.

[femtomc]

haha, that's the spirit! Again, no worries if you have to revert.

[pawamoy]

Let me know if v1.8.2 works better for you 🙂

[femtomc]

Re -- @pawamoy, if you have some time, can you examine the response by Cecil over at beartype/beartype#381 (comment) (the responses from Cecil on the repo are in a great style)

There's one section in particular which I'm wondering about:

When the Badness Get Going, the Code Get Blowing Up

Are the things that Cecil is saying in this section true of markdown-exec -- in particular:

Destroying type hints by stringifying usable type hints into unusable strings. How? Presumably, by enabling PEP 563. How? Presumably, by forcefully injecting from future import annotations at the top of your module without your permission.

I believe this might also explain some weird omissions in my print(...) statements with types using markdown-exec, which we can discuss in a separate issue.

[pawamoy]

As answered in the linked issue, it's possible that my own import of future annotations is leaking into the context of the executed code block. Were you able to try v1.8.2 though? I'm not sure to understand if you have a new error or if this works now.