Mermaid Crashes If trying to draw a large pipeline #8089
Description
Thanks in advance for your help :)
Describe the bug
I was building a huge pipeline, 30 components and 35 connections, and for debugging proposes I wanted to display the diagram, but both .draw() and .show() methods failed. It still works with small pipelines by the way.
Error message
Failed to draw the pipeline: https://mermaid.ink/img/ returned status 400
No pipeline diagram will be saved.
Failed to draw the pipeline: could not connect to https://mermaid.ink/img/ (400 Client Error: Bad Request for url: https://mermaid.ink/img/{place holder for 2km long data}
No pipeline diagram will be saved.
Traceback (most recent call last):
File "/Users/carlosfernandezloran/Desktop/babyagi-classic-haystack/.venv/lib/python3.10/site-packages/haystack/core/pipeline/draw.py", line 87, in _to_mermaid_image
resp.raise_for_status()
File "/Users/carlosfernandezloran/Desktop/babyagi-classic-haystack/.venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://mermaid.ink/img/{another placeholder}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/carlosfernandezloran/Desktop/babyagi-classic-haystack/babyagi.py", line 188, in <module>
pipe.draw(path=Path("pipe"))
File "/Users/carlosfernandezloran/Desktop/babyagi-classic-haystack/.venv/lib/python3.10/site-packages/haystack/core/pipeline/base.py", line 649, in draw
image_data = _to_mermaid_image(self.graph)
File "/Users/carlosfernandezloran/Desktop/babyagi-classic-haystack/.venv/lib/python3.10/site-packages/haystack/core/pipeline/draw.py", line 95, in _to_mermaid_image
raise PipelineDrawingError(
haystack.core.errors.PipelineDrawingError: There was an issue with https://mermaid.ink/, see the stacktrace for details.
Expected behavior
I expect the .show() and .draw() methods to work for all pipelines, no matter the size.
This might be a Mermaid problem and not strictly haystacks, but we would need to work to implement a local diagram generator as said in #7896
To Reproduce
I will not add all the 200 lines of add_component, connect statements, but you can imagine how it goes.
System:
- OS: macOS
- GPU/CPU: M1
- Haystack version (commit or version number): 2.3.0