Skip to content

Commit 1639fc6

Browse files
localai-botmudler
andauthored
chore(model gallery): 🤖 add 1 new models via gallery agent (#7831)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <[email protected]>
1 parent 841e8f6 commit 1639fc6

File tree

1 file changed

+37
-0
lines changed

1 file changed

+37
-0
lines changed

gallery/index.yaml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,41 @@
11
---
2+
- name: "iquest-coder-v1-40b-instruct-i1"
3+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
4+
urls:
5+
- https://huggingface.co/mradermacher/IQuest-Coder-V1-40B-Instruct-i1-GGUF
6+
description: |
7+
The **IQuest-Coder-V1-40B-Instruct-i1-GGUF** is a quantized version of the original **IQuestLab/IQuest-Coder-V1-40B-Instruct** model, designed for efficient deployment. It is an **instruction-following large language model** with 40 billion parameters, optimized for tasks like code generation and reasoning.
8+
9+
**Key Features:**
10+
- **Size:** 40B parameters (quantized for efficiency).
11+
- **Purpose:** Instruction-based coding and reasoning.
12+
- **Format:** GGUF (supports multi-part files).
13+
- **Quantization:** Uses advanced techniques (e.g., IQ3_M, Q4_K_M) for balance between performance and quality.
14+
15+
**Available Quantizations:**
16+
- Optimized for speed and size: **i1-Q4_K_M** (recommended).
17+
- Lower-quality options for trade-off between size/quality.
18+
19+
**Note:** This is a **quantized version** of the original model, but the base model (IQuestLab/IQuest-Coder-V1-40B-Instruct) is the official source. For full functionality, use the unquantized version or verify compatibility with your deployment tools.
20+
overrides:
21+
parameters:
22+
model: llama-cpp/models/IQuest-Coder-V1-40B-Instruct.i1-Q4_K_M.gguf
23+
name: IQuest-Coder-V1-40B-Instruct-i1-GGUF
24+
backend: llama-cpp
25+
template:
26+
use_tokenizer_template: true
27+
known_usecases:
28+
- chat
29+
function:
30+
grammar:
31+
disable: true
32+
description: Imported from https://huggingface.co/mradermacher/IQuest-Coder-V1-40B-Instruct-i1-GGUF
33+
options:
34+
- use_jinja:true
35+
files:
36+
- filename: llama-cpp/models/IQuest-Coder-V1-40B-Instruct.i1-Q4_K_M.gguf
37+
sha256: 0090b84ea8e5a862352cbb44498bd6b4cd38564834182813c35ed84209050b51
38+
uri: https://huggingface.co/mradermacher/IQuest-Coder-V1-40B-Instruct-i1-GGUF/resolve/main/IQuest-Coder-V1-40B-Instruct.i1-Q4_K_M.gguf
239
- name: "onerec-8b"
340
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
441
urls:

0 commit comments

Comments
 (0)