Skip to content

Ensure DAG pipeline isolates node inputs#10

Open
robitec97 wants to merge 3 commits intodatapizza-labs:mainfrom
robitec97:main
Open

Ensure DAG pipeline isolates node inputs#10
robitec97 wants to merge 3 commits intodatapizza-labs:mainfrom
robitec97:main

Conversation

@robitec97
Copy link

Hello,
I've tried to fix a few things mentioned in your issue: #2

  • Ensured _get_args_for_node deep copies dependency values so DAG pipeline nodes receive isolated inputs rather than shared references.

  • Added a regression test confirming branches that share a source node get independent copies even when one consumer mutates its input.

Testing

  • pytest datapizza-ai-core/datapizza/pipeline/tests/test_graph_pipeline.py

@AndreaBozzo
Copy link
Contributor

Hi Roby, i noticed you referenced the Issue #2.

I avoided submitting a PR on this as i have no real Infos of the future intents about deepcopy usage, as Is pretty damn expensive on performances.

Still waiting on a clarification from dp Dev team on It.

@robitec97
Copy link
Author

Hello Andrea, you’re absolutely right. I’m so sorry! :) I’ll remove my PR, I really didn’t mean to take any credit.

@robitec97 robitec97 closed this Oct 13, 2025
@AndreaBozzo
Copy link
Contributor

@robitec97 Sorry Bro i didn't mean to remove the PR, i Just meant that i needed more Infos about that from the Dev team, i have absolutely no problems with you submitting a PR for Issue #2, i Just wanted to add more context.

Your PR is legit, keep It open imo

@robitec97 robitec97 reopened this Oct 13, 2025
@Federaffo
Copy link
Contributor

We haven't added deepcopy yet because we get an error while deepcopying the memory returned by ChatPromptTemplate
You can reproduce the error running a DagPipeline

"TypeError: Template.new() missing 1 required positional argument: 'source'"

@AndreaBozzo
Copy link
Contributor

Grazie @Federaffo for the hint, i'll be looking on this.

Constructors apparently do not accept the empty invocation, could possibly use a more complex template to handle/avoid deepcopying the Jinja templates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants