code block exited with non-zero status does not provide enough information for addressing the issue

[ssbarnea]

Describe the bug
When execution of a bash code block fails, there is not context information provided, making impossible to debug the issue (which can also happen on CI).

WARNING  -  �[0mmarkdown_exec: Execution of bash code block exited with non-zero status

Considering that I have 10+ command execution inside docs, finding the one that failed on CI become a gamble.

Expected behavior

• display filename causing the failure
• display command that returned the error, including the returned error
• display both stdout and stderr output, so user can figure-out the build failure

Screenshots

System (please complete the following information):

• Markdown Exec version: [e.g. 0.2.1]
• Python version: [e.g. 3.8]
• OS: [Windows/Linux]

Additional context
Add any other context about the problem here.

[pawamoy]

I've recently added a paragraph about handling errors: https://pawamoy.github.io/markdown-exec/usage/#handling-errors. We actually provide an id option that allows to identify code blocks when there's an issue. Similarly, if you use the title option (used by Material for MkDocs), it serves as an id as well.

[pawamoy]

To get back on the things you're suggesting:

display filename causing the failure

Not possible unfortunately, as this project works as a Markdown extension, and therefore doesn't have access to MkDocs data when running (i.e. filenames).

display command that returned the error, including the returned error

Possible, but probably too verbose for logs, since code blocks can be heavy.

display both stdout and stderr output, so user can figure-out the build failure

That should be the case for shell code blocks. Let me check.

[pawamoy]

Yep, stdout and stderr are combined:

markdown-exec/src/markdown_exec/formatters/sh.py

Lines 11 to 16 in 861dad0

	def _run_sh(code: str, **extra: str) -> str:
	try:
	output = subprocess.check_output(["sh", "-c", code], stderr=subprocess.STDOUT).decode() # noqa: S603,S607
	except subprocess.CalledProcessError as error:
	raise RuntimeError(code_block("sh", error.output, **extra))
	return output

But this is displayed in the rendered docs indeed, not in the logs.

[pawamoy]

I guess it would be fine to log more things, even if very verbose, since errors are the outliers, not the common case.

[ssbarnea]

Exactly, logging will happen only on case of failure/warning; not when everything is "as expected".

[pawamoy]

WDYT of this?

INFO     -  Building documentation...
INFO     -  Cleaning site directory
INFO     -  UserWarning: stdlib_list does't support Python 3.10 yet, pydeps will use symbols from 3.9 for now.
              File "/media/data/dev/markdown-exec/__pypackages__/3.10/lib/pydeps/py2depgraph.py", line 249, in py2dep
                pylib = pystdlib()
              File "/media/data/dev/markdown-exec/__pypackages__/3.10/lib/pydeps/pystdlib.py", line 17, in pystdlib
                warnings.warn(
WARNING  -  markdown_exec: Execution of bash code block exited with unexpected code 1

            Code block is:

              echo hello mon pote
              exit 1

            Output is:

              hello mon pote

WARNING  -  markdown_exec: Execution of python code block exited with errors

            Code block is:

              print("hello again my friend")
              print(1 / 0)

            Output is:

              Traceback (most recent call last):
                File "/media/data/dev/markdown-exec/src/markdown_exec/formatters/python.py", line 23, in _run_python
                  exec(code, exec_globals)  # noqa: S102
                File "<executed code block>", line 2, in <module>
                  print(1 / 0)
              ZeroDivisionError: division by zero

INFO     -  Documentation built in 4.96 seconds
INFO     -  [12:52:27] Watching paths for changes: 'docs', 'mkdocs.yml', 'README.md', 'CONTRIBUTING.md', 'CHANGELOG.md', 'src/markdown_exec'
INFO     -  [12:52:27] Serving on http://127.0.0.1:8000/markdown-exec/

[ssbarnea]

I love it. I would personally have avoided the "-" column while logging but I guess that this has nothing to do with the introduced warnings. These are exactly what I was hoping to see during build, so I can step-in and fix the code.

I understand the markdown filename is not available but printing the block is enough to provide something to copy and paste into search box/cli.

[pawamoy]

Great, I'll commit, push and probably release today 🙂
Yeah, the "-" column comes from MkDocs itself.