Skip to content

Conversation

@Inokinoki
Copy link
Member

Main points:

  • add ChatTestCasesResource and ChatTestCase data class
  • mark conversation usage as deprecated with warnings
  • get chat_test_case or conversation from the evaluation results to ensure compatibility

@linear
Copy link

linear bot commented May 20, 2025

@Inokinoki Inokinoki self-assigned this May 20, 2025
@Inokinoki Inokinoki requested a review from henchaves May 20, 2025 12:37
@henchaves
Copy link
Member

hey @Inokinoki, it seems very good :)

could you just also add some tests like:

  • creating a ChatTestCase and a Conversation and checking if their dict content are the same
  • listing ChatTestCase from evaluations and datasets and checking if everything is fine

@Inokinoki
Copy link
Member Author

hey @Inokinoki, it seems very good :)

could you just also add some tests like:

  • creating a ChatTestCase and a Conversation and checking if their dict content are the same
  • listing ChatTestCase from evaluations and datasets and checking if everything is fine

Hey, during adding the tests, I found that the checks don't seem to have the right type after _format_checks_to_cli: https://github.com/Giskard-AI/giskard-hub/pull/52/files#diff-cc6693ba3e3ac6afcac5f7de2111269e4b3248bf3bb2c188db53ddee49c05f8eR77

It's still dict while we are expecting TestCaseCheckConfig as the arg of _format_checks_to_cli and CheckConfig as the return value, as well in the checks field.

Is this something wrong? How should we process them if so, will anything break if we also fix it here?

@Inokinoki
Copy link
Member Author

hey @Inokinoki, it seems very good :)
could you just also add some tests like:

  • creating a ChatTestCase and a Conversation and checking if their dict content are the same
  • listing ChatTestCase from evaluations and datasets and checking if everything is fine

Hey, during adding the tests, I found that the checks don't seem to have the right type after _format_checks_to_cli: https://github.com/Giskard-AI/giskard-hub/pull/52/files#diff-cc6693ba3e3ac6afcac5f7de2111269e4b3248bf3bb2c188db53ddee49c05f8eR77

It's still dict while we are expecting TestCaseCheckConfig as the arg of _format_checks_to_cli and CheckConfig as the return value, as well in the checks field.

Is this something wrong? How should we process them if so, will anything break if we also fix it here?

To be fixed in https://github.com/Giskard-AI/giskard-hub/compare/feature/gsk-4163-add-missing-resources

@Inokinoki
Copy link
Member Author

Thanks @henchaves for discovering the URLs that we have already changed

@Inokinoki Inokinoki requested a review from henchaves June 13, 2025 13:42
@sonarqubecloud
Copy link

Copy link
Member

@henchaves henchaves left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me! Thanks for all the hard work, @Inokinoki

@henchaves henchaves merged commit 7de7cd4 into main Jun 13, 2025
8 checks passed
@henchaves henchaves deleted the feature/gsk-4308-rename-any-conversation-mention-to-chattestcase-testcase branch June 13, 2025 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants