Skip to content

Set any groq model as default LLMClient for giskard  #1977

@Vikas9758

Description

@Vikas9758

🚀 Feature Request

Add Support for Groq in Giskard or show it in the documentation if already supported

🔈 Motivation

The integration of Groq as an alternative to OpenAI in the Giskard library is highly beneficial. Groq offers speed that is almost equivalent or even faster than GPT-4 and significantly faster than any Ollama model. This will provide users with a high-performance alternative for their machine learning tasks.

🛰 Alternatives

Currently, the only alternative is using OpenAI's models, which may not meet all users' needs for speed and performance. Can use Ollama as well but using it is a lot of hassle especially in colab or kaggle notebooks.Incorporating Groq provides a competitive edge and additional flexibility.

📎 Additional context

Here is the proposed code to integrate Groq with Giskard:

from groq import `Groq`
from giskard.llm.client.openai import OpenAIClient
import giskard as gsk

_client = Groq(api_key="YOUR_GROQ_API_KEY")
oc = OpenAIClient(model="any_model_in_groq", client=_client)
gsk.llm.set_default_client(oc)

This simple addition allows users to set up Groq as their default client in Giskard, enabling them to take advantage of Groq's superior speed and performance. Detailed documentation and examples will also be necessary to guide users through the setup and usage process.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions