It is possible to add arbitrary HTML to cell outputs via:
from IPython.core.display import display, HTML
display(HTML('<h2 id="aheader">hello!</h2>'))
These headers are added automatically to the TOC sidebar in jupyter lab, but are not in the TOC of exported HTML documents. It would be very convenient if they did!
A possible way of doing this is to change the selector here to include HTML output cells which have class .output_html:
all_headers = $('.text_cell_render,.output_html').find('[id]:header:not(:has(.tocSkip))');