[Feature] Enable remote inference endpoints for examples

[joshuayao]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

Story for users

1. for Xeon users, users can try bigger models on remote endpoint with Gaudi while bigger models might not work well on Xeon
2. for Xeon users, if users want to try on Gaudi to understand the difference between Xeon and Gaudi, they can use remote endpoint
3. for Gaudi users, we assume that users have local Gaudi access to run GenAIExamples.
4. target Denvr and IBM for remote endpoints in the GenAIExamples

Enable remote inference endpoints for

• AgentQnA : Sin, Alex

  • AgentQnA - add support for remote server #1900
  • Agent: add support for remote server GenAIComps#1644
  • AgentQnA: update instructions and env variable names for remote endpoints #2113

• Productivity Suite : (Sri?) (P0)

  • Support remote endpoints  #2085
  • Enable remote inference non streaming response GenAIComps#1806

• ChatQnA : (Sri?) (P0)

  • Support remote endpoints  #2085
  • Enable remote inference non streaming response GenAIComps#1806

• DocSum : (Sri?) (P0)

  • Support remote endpoints  #2085
  • Enable remote inference non streaming response GenAIComps#1806

• CodeGen : (Sri?) (P0)

  • Support remote endpoints  #2085
  • Enable remote inference non streaming response GenAIComps#1806

• FinanceAgent : Alex

  • FinanceAgent - enable on Xeon, remote endpoint, and refactor tests #2032

• workflowExecAgent : Louie

  • Fix workexec agent docker build issues and enable LLM Remote Endpoint #2103

• CodeTrans : Alex

  • Codetrans: enable remote endpoints #2195

• AudioQnA : Alex

  • Audioqna: fix input for TTS service, enable remote endpoints #2101

Models are not supported in current public remote endpoint. ON HOLD for now.

• VideoQnA : (?)
• MulitmodalQnA : Alex
• VisualQnA: Alex

[louie-tsai]

@alexsin368 needs to be in assignees list

[joshuayao]

@alexsin368 needs to be in assignees list

@louie-tsai Ok, I will add @alexsin368 to the repo. Then, we can add him into the assignees list.
@alexsin368 Could you please accept the repo invitation in GitHub?

[alexsin368]

accepted, thanks