Skip to content

Add Tag-bench in agent_eval#230

Merged
lvliang-intel merged 12 commits intoopea-project:mainfrom
minmin-intel:tag-bench-dev
Apr 3, 2025
Merged

Add Tag-bench in agent_eval#230
lvliang-intel merged 12 commits intoopea-project:mainfrom
minmin-intel:tag-bench-dev

Conversation

@minmin-intel
Copy link
Copy Markdown
Collaborator

@minmin-intel minmin-intel commented Jan 31, 2025

Description

  1. Add TAG-bench for evaluating SQL agents.
  2. reorg agent_eval folder now that we have two benchmarks. extract out common elements like how to launch vllm-gaudi. and added a new readme.
  3. update crag_eval given the new changes in OPEA agent code.

Issues

#224

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

no new ones

Tests

All commands in readme were tested locally.

@minmin-intel
initial code for tag-bench
d45072a 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
add vllm endpoint for ragas and update tag-bench scripts
df1596c 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
update readme and test scripts
e1bbbe9 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
refactor crag_eval
4e1deca 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
update scripts for tag-bench generate answers
de5d960 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
update code given recent changes in OPEA agent
d725ce3 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
@minmin-intel
Merge branch 'main' into tag-bench-dev
3fbf261
@joshuayao joshuayao linked an issue Feb 10, 2025 that may be closed by this pull request
[Feature] add TAG-bench for agent_eval, publish TAG-bench results for sql agent llama #224
Closed
1 task
@joshuayao joshuayao mentioned this pull request Mar 10, 2025
Closed
1 task
@joshuayao joshuayao requested a review from lvliang-intel April 3, 2025 00:56
lkk12014402
lkk12014402 reviewed Apr 3, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@lkk12014402 lkk12014402 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

lvliang-intel
lvliang-intel approved these changes Apr 3, 2025
View reviewed changes
@lvliang-intel lvliang-intel merged commit 69e018a into opea-project:main Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lvliang-intel lvliang-intel lvliang-intel approved these changes

@lkk12014402 lkk12014402 lkk12014402 approved these changes

@XinyuYe-Intel XinyuYe-Intel Awaiting requested review from XinyuYe-Intel XinyuYe-Intel is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature] add TAG-bench for agent_eval, publish TAG-bench results for sql agent llama

3 participants

@minmin-intel @lkk12014402 @lvliang-intel