Skip to content

Conversation

@kevinmessiaen
Copy link
Member

Description

Introduce not given and apply it to dataset's target

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've updated the code style using make codestyle.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in Google format for all the methods and classes that I used.

@linear
Copy link

linear bot commented Nov 15, 2023

@kevinmessiaen kevinmessiaen marked this pull request as ready for review November 16, 2023 10:35
Copy link
Contributor

@rabah-khalek rabah-khalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me @kevinmessiaen.

Could we also undo the changes done in https://github.com/Giskard-AI/giskard/pull/1531/files (the ones related to suppress the warning of target for LLM) and add the target=None in the BaseModel for when model_type == "text_generation" to suppress the warning?

@kevinmessiaen
Copy link
Member Author

kevinmessiaen commented Nov 20, 2023

Looks good to me @kevinmessiaen.

Could we also undo the changes done in https://github.com/Giskard-AI/giskard/pull/1531/files (the ones related to suppress the warning of target for LLM) and add the target=None in the BaseModel for when model_type == "text_generation" to suppress the warning?

Not sure to understand the logic of the validation though, why to we validate the dataset target when validating model?

For me this validation is completely unrelated to the model validation and should be done when validating dataset only.
When uploading a model we should not confuse the user with warning that the dataset is missing a target, since it's unrelated.

@rabah-khalek
Copy link
Contributor

that's exactly what I meant by my comment. We should remove it.
We had to do it to suppress the warning only for LLMs. We needed to do it inside validate_model just to know the model_type back then. But now people can put explicitly target=None to supress this warning

@kevinmessiaen
Copy link
Member Author

that's exactly what I meant by my comment. We should remove it. We had to do it to suppress the warning only for LLMs. We needed to do it inside validate_model just to know the model_type back then. But now people can put explicitly target=None to supress this warning

Thanks for clarification, so it's what I did then, you can review the PR when you have time

@kevinmessiaen kevinmessiaen self-assigned this Nov 27, 2023
@kevinmessiaen kevinmessiaen removed their assignment Nov 27, 2023
kevinmessiaen and others added 2 commits November 28, 2023 09:09
# Conflicts:
#	docs/reference/notebooks/LLM_QA_Google.ipynb
#	giskard/core/model_validation.py
Copy link
Contributor

@rabah-khalek rabah-khalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kevinmessiaen, looks good

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

96.3% 96.3% Coverage
0.0% 0.0% Duplication

@kevinmessiaen kevinmessiaen merged commit 204c1b0 into main Nov 29, 2023
@kevinmessiaen kevinmessiaen deleted the feature/gsk-2118-introduce-not_given-type-2 branch November 29, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants