-
Notifications
You must be signed in to change notification settings - Fork 31.4k
Allow RAG to output decoder cross-attentions #9789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow RAG to output decoder cross-attentions #9789
Conversation
lhoestq
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks ! Looks good to me :)
The suggestions below should help you fix the CI.
I think it just comes from a line length issue
Co-authored-by: Quentin Lhoest <[email protected]>
|
@lhoestq Thanks for the suggestions! All the CI checks pass now. |
lhoestq
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice !
Also pinging @patrickvonplaten just to make sure
patrickvonplaten
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super! Thanks for you contribution
What does this PR do?
This PR makes RAG output the generator model's decoder cross-attentions when
output_attentions=True.Motivation and context: before this PR, RAG's output objects had attributes for the generator's encoder self-attentions and decoder self-attentions, but no option for the encoder-decoder cross-attentions. So this simply allows cross-attentions to be extracted, as well as fixing a small bug where
output_attentionswasn't being passed into the generator.Fixes #9468
Before submitting
Pull Request section?
to it if that's the case. Yes - Have RAG return generator cross-attentions when output_attentions=True #9468
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@patrickvonplaten, @lhoestq