Skip to content

Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi#1316

Merged
mkbhanda merged 3 commits intoopea-project:mainfrom
intel-ai-tce:otlp_chatqna_tgi
Jan 9, 2025
Merged

Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi#1316
mkbhanda merged 3 commits intoopea-project:mainfrom
intel-ai-tce:otlp_chatqna_tgi

Conversation

@louie-tsai
Copy link
Copy Markdown
Collaborator

@louie-tsai louie-tsai commented Dec 27, 2024 •
edited
Loading

Description

Enable Jaeger UI and OpenTelemetry tracing for ChatQnA TGI serving on Gaudi.
user could see OPEA, TEI, and TGI open telemetry tracing on Jaeger UI.

Screenshot from 2024-12-27 11-58-18

Screenshot from 2024-12-27 11-26-25

Screenshot from 2024-12-27 11-25-30

PR#1122 depends on this PR

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

Jaeger docker image

Tests

Manually testing on Gaudi

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 27, 2024 •
edited
Loading

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

@louie-tsai louie-tsai requested a review from Spycsh December 27, 2024 20:58
@chensuyue
Copy link
Copy Markdown
Collaborator

chensuyue commented Jan 2, 2025

Please fix the issue reported by CI.

@louie-tsai
Copy link
Copy Markdown
Collaborator Author

louie-tsai commented Jan 2, 2025

Please fix the issue reported by CI.

addressed pre-commit and merge-conflict. should I take care missing dockerfile issue? it doesn't seem to be related to this PR.

@chensuyue
Copy link
Copy Markdown
Collaborator

chensuyue commented Jan 3, 2025

Please fix the issue reported by CI.

addressed pre-commit and merge-conflict. should I take care missing dockerfile issue? it doesn't seem to be related to this PR.

Please merge main branch into yours, and the issues will disappear.

Spycsh
Spycsh reviewed Jan 3, 2025
View reviewed changes
Spycsh
Spycsh reviewed Jan 3, 2025
View reviewed changes
louie-tsai
louie-tsai commented Jan 3, 2025
View reviewed changes
fidencio
fidencio reviewed Jan 3, 2025
View reviewed changes
fidencio
fidencio reviewed Jan 3, 2025
View reviewed changes
ftian1
ftian1 reviewed Jan 6, 2025
View reviewed changes
@louie-tsai louie-tsai requested review from fidencio and ftian1 January 6, 2025 05:15
@louie-tsai louie-tsai requested a review from letonghan as a code owner January 6, 2025 19:48
@chensuyue
Copy link
Copy Markdown
Collaborator

chensuyue commented Jan 7, 2025

Please check the CI issue.

@louie-tsai
Copy link
Copy Markdown
Collaborator Author

louie-tsai commented Jan 7, 2025

Please check the CI issue.

@chensuyue I rebased to the latest codes couple time, but still have dockerfile test issue. any suggestion?

@louie-tsai
Copy link
Copy Markdown
Collaborator Author

louie-tsai commented Jan 7, 2025 •
edited
Loading

Please check the CI issue.

@chensuyue
Final issue is related to --otlp-endpoint, but the value is provided in set_env.sh.

image

also docker compose doesn't get the export OTLP env variable from set_env.sh
image
Does CI also source the new set_env.sh in the PR?

CI doesn't use set_env.sh. I added those env variable into test script instead.

@louie-tsai louie-tsai force-pushed the otlp_chatqna_tgi branch 2 times, most recently from 551d966 to 551b9e3 Compare January 7, 2025 08:57
@louie-tsai
Enable OpenTelemetry Tracing for ChatQnA on Gaudi with TGI serving
9d317b8 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
@louie-tsai
fix for CI
9383f72 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
@louie-tsai louie-tsai requested review from fidencio, preethivenkatesh and yinghu5 and removed request for fidencio January 7, 2025 18:55
eero-t
eero-t reviewed Jan 7, 2025
View reviewed changes
@louie-tsai louie-tsai mentioned this pull request Jan 7, 2025
Merged
4 tasks
@louie-tsai
removed privilege mode due to security concern
003995b 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
@louie-tsai louie-tsai requested a review from eero-t January 9, 2025 00:53
mkbhanda
mkbhanda approved these changes Jan 9, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@mkbhanda mkbhanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @louie-tsai for addressing comments.

@mkbhanda mkbhanda merged commit 8102235 into opea-project:main Jan 9, 2025
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Jan 15, 2025
@louie-tsai
Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi (opea-p…
f0db9b9 
…roject#1316)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
@joshuayao joshuayao mentioned this pull request Feb 13, 2025
Closed
9 tasks
@louie-tsai louie-tsai mentioned this pull request Mar 6, 2025
Closed
@joshuayao joshuayao added this to the v1.3 milestone Mar 7, 2025
cogniware-devops pushed a commit to Cogniware-Inc/GenAIExamples that referenced this pull request Dec 19, 2025
@louie-tsai @cogniware-devops
Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi (opea-p…
873600a 
…roject#1316)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: cogniware-devops <ambarish.desai@cogniware.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mkbhanda mkbhanda mkbhanda approved these changes

@Spycsh Spycsh Spycsh approved these changes

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

@chensuyue chensuyue Awaiting requested review from chensuyue

@ftian1 ftian1 Awaiting requested review from ftian1

@letonghan letonghan Awaiting requested review from letonghan

@preethivenkatesh preethivenkatesh Awaiting requested review from preethivenkatesh

@yinghu5 yinghu5 Awaiting requested review from yinghu5

@fidencio fidencio Awaiting requested review from fidencio

@eero-t eero-t Awaiting requested review from eero-t

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v1.3

Development

Successfully merging this pull request may close these issues.

8 participants

@louie-tsai @chensuyue @fidencio @mkbhanda @eero-t @ftian1 @Spycsh @joshuayao