Skip to content

Added a LLM-as-judge per row evaluator #1672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Dec 20, 2023
Merged

Added a LLM-as-judge per row evaluator #1672

merged 20 commits into from
Dec 20, 2023
+229 −51

Conversation

@kevinmessiaen
Copy link
Member

@kevinmessiaen kevinmessiaen commented Dec 7, 2023 •
edited
Loading

Description

Added a LLM-as-judge per row evaluator

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

@linear
Copy link

linear bot commented Dec 7, 2023

GSK-2300 LLM-as-a-judge row-wise custom requirement test

@kevinmessiaen kevinmessiaen requested review from mattbit and rabah-khalek and removed request for mattbit December 7, 2023 10:43
@kevinmessiaen kevinmessiaen marked this pull request as draft December 7, 2023 10:46
@kevinmessiaen kevinmessiaen marked this pull request as ready for review December 7, 2023 14:21
andreybavt
andreybavt reviewed Dec 14, 2023
View reviewed changes
andreybavt
andreybavt reviewed Dec 14, 2023
View reviewed changes
Copy link
Contributor

@andreybavt andreybavt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apart from the output_df question looks good to me

@Hartorn Hartorn enabled auto-merge (squash) December 20, 2023 15:29
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 20, 2023

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
97.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@Hartorn Hartorn merged commit 1e1d1a8 into main Dec 20, 2023
@Hartorn Hartorn deleted the feature/gsk-2300-llm-as-a-judge-row-wise-custom-requirement-test branch December 20, 2023 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rabah-khalek rabah-khalek Awaiting requested review from rabah-khalek

@mattbit mattbit Awaiting requested review from mattbit

@Hartorn Hartorn Awaiting requested review from Hartorn

1 more reviewer

@andreybavt andreybavt andreybavt approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kevinmessiaen @andreybavt @Hartorn