feat: add support for local llms running in cluster

[pauldotyu]

Purpose

• This enhancement is to add support for LLMs running locally in the cluster (i.e., models and inferencing endpoints deployed by KAITO).
• To use local LLMs, you will need to set the USE_LOCAL_LLM environment variable in the ai-service.yaml manifest to "True" and AI_ENDPOINT to the inferencing service that is running in the cluster

Does this introduce a breaking change?

[ ] Yes
[x] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

• Ensure this doc still works https://learn.microsoft.com/en-us/azure/aks/open-ai-quickstart?tabs=aoai
• To test local LLM, deploy KAITO to AKS cluster and update ai-service.yaml manifest to set USE_LOCAL_LLM to "True" and AI_ENDPOINT to the inferencing service that KAITO configures. Also be sure to unset the OpenAI specific environment variables (e.g., USE_AZURE_OPENAI, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_DEPLOYMENT_NAME, OPENAI_API_KEY, and OPENAI_ORG_ID in the ai-service.yaml manifest)

[chzbrgr71]

Completed some basic local testing. Will confirm again once new image is created