Skip to content

Commit daf69f3

Browse files
nekomeowwwnekomeowww
authored
feat: refactors, added new extraEnvVar, extraEnvFrom, runtimeClassName and podTemplate for Model CRD, fixes (#18)
- Refactored controller with new wrappers and utilities - Fixed curl not working in the container - Added new fields `extraEnvVar`, `extraEnvFrom`, `runtimeClassName` and `podTemplate` for Model CRD Close #15, #16, #17 Signed-off-by: Neko Ayaka <[email protected]>
1 parent 2717e82 commit daf69f3

10 files changed

Lines changed: 8200 additions & 128 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -145,34 +145,8 @@ Unlock the abilities to run the following models with the Ollama Operator over K
145145
146146
> [!TIP]
147147
> By the power of [`Modelfile`](https://github.com/ollama/ollama/blob/main/docs/modelfile.md) backed by Ollama, you can create and bundle any of your own model. **As long as it's a GGUF formatted model.**
148-
149-
| Model | Parameters | Size | Model image | Full model image URL | Multi-modal | Uncensored |
150-
| ------------------------------------------------------------------ | ---------- | ----- | ------------------- | ---------------------------------------------- | ----------- | ---------- |
151-
| [Phi-3 Mini](https://ollama.com/library/phi3) | 3.8B | 2.3GB | `phi3` | `registry.ollama.ai/library/phi3` | | |
152-
| [Llama 3](https://ollama.com/library/llama3) | 8B | 4.7GB | `llama3` | `registry.ollama.ai/library/llama3` | | |
153-
| [Dolphin Llama 3](https://ollama.com/library/dolphin-llama3) | 8B | 4.7GB | `dolphin-llama3` | `registry.ollama.ai/dolphin-llama3` | | ✅ |
154-
| [WizardLM-2](https://ollama.com/library/wizardlm2) | 7B | 4.1GB | `wizardlm2` | `registry.ollama.ai/library/wizardlm2` | | |
155-
| [Llama 2](https://ollama.com/library/llama2) | 7B | 3.8GB | `llama2` | `registry.ollama.ai/library/llama2` | | |
156-
| [Mistral](https://ollama.com/library/mistral) | 7B | 4.1GB | `mistral` | `registry.ollama.ai/library/mistral` | | |
157-
| [Mixtral 8x7B](https://ollama.com/library/mixtral:8x7b) | 8x7B | 26GB | `mixtral:8x7b` | `registry.ollama.ai/library/mixtral:8x7b` | | |
158-
| [Mixtral 8x22B](https://ollama.com/library/mixtral:8x22b) | 8x22B | 80GB | `mixtral:8x22b` | `registry.ollama.ai/library/mixtral:8x22b` | | |
159-
| [Command R](https://ollama.com/library/command-r) | 35B | 20GB | `command-r` | `registry.ollama.ai/library/command-r` | | |
160-
| [Command R Plus](https://ollama.com/library/command-r-plus) | 104B | 59GB | `command-r-plus` | `registry.ollama.ai/library/command-r-plus` | | |
161-
| [Dolphin Phi](https://ollama.com/library/dolphin-phi) | 2.7B | 1.6GB | `dolphin-phi` | `registry.ollama.ai/library/dolphin-phi` | | ✅ |
162-
| [Phi-2](https://ollama.com/library/phi) | 2.7B | 1.7GB | `phi` | `registry.ollama.ai/library/phi` | | |
163-
| [Neural Chat](https://ollama.com/library/neural-chat) | 7B | 4.1GB | `neural-chat` | `registry.ollama.ai/library/neural-chat` | | |
164-
| [Starling](https://ollama.com/library/starling-lm) | 7B | 4.1GB | `starling-lm` | `registry.ollama.ai/library/starling-lm` | | |
165-
| [Code Llama](https://ollama.com/library/codellama) | 7B | 3.8GB | `codellama` | `registry.ollama.ai/library/codellama` | | |
166-
| [Llama 2 Uncensored](https://ollama.com/library/llama2-uncensored) | 7B | 3.8GB | `llama2-uncensored` | `registry.ollama.ai/library/llama2-uncensored` | | ✅ |
167-
| [Llama 2 13B](https://ollama.com/library/llama2) | 13B | 7.3GB | `llama2:13b` | `registry.ollama.ai/library/llama2:13b` | | |
168-
| [Llama 2 70B](https://ollama.com/library/llama2) | 70B | 39GB | `llama2:70b` | `registry.ollama.ai/library/llama2:70b` | | |
169-
| Orca Mini | 3B | 1.9GB | `orca-mini` | `registry.ollama.ai/library/orca-mini` | | |
170-
| Vicuna | 7B | 3.8GB | `vicuna` | `registry.ollama.ai/library/vicuna` | | |
171-
| LLaVA | 7B | 4.5GB | `llava` | `registry.ollama.ai/library/llava` | ✅ | |
172-
| Gemma 2B | 2B | 1.4GB | `gemma:2b` | `registry.ollama.ai/library/gemma:2b` | | |
173-
| Gemma 7B | 7B | 4.8GB | `gemma:7b` | `registry.ollama.ai/library/gemma:7b` | | |
174-
175-
Full list of available images can be found at [Ollama Library](https://ollama.com/library).
148+
>
149+
> Full list of available images can be found at [Ollama Library](https://ollama.com/library).
176150
177151
> [!WARNING]
178152
> You should have at least 8 GB of RAM available on your node to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

‎api/ollama/v1/model_types.go‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,30 @@ type ModelSpec struct {
7878
// More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes
7979
// +optional
8080
PersistentVolume *ModelPersistentVolumeSpec `json:"persistentVolume,omitempty" protobuf:"bytes,8,opt,name=persistentVolume"`
81+
// RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used
82+
// to run this pod. If no RuntimeClass resource matches the named class, the pod will not be run.
83+
// If unset or empty, the "legacy" RuntimeClass will be used, which is an implicit class with an
84+
// empty definition that uses the default runtime handler.
85+
// More info: https://git.k8s.io/enhancements/keps/sig-node/585-runtime-class
86+
// +optional
87+
RuntimeClassName *string `json:"runtimeClassName,omitempty" protobuf:"bytes,9,opt,name=runtimeClassName"`
88+
// List of sources to populate environment variables in the container.
89+
// The keys defined within a source must be a C_IDENTIFIER. All invalid keys
90+
// will be reported as an event when the container is starting. When a key exists in multiple
91+
// sources, the value associated with the last source will take precedence.
92+
// Values defined by an Env with a duplicate key will take precedence.
93+
// Cannot be updated.
94+
// +optional
95+
ExtraEnvFrom []corev1.EnvFromSource `json:"extraEnvFrom,omitempty" protobuf:"bytes,10,rep,name=extraEnvFrom"`
96+
// List of environment variables to set in the container.
97+
// Cannot be updated.
98+
// +optional
99+
// +patchMergeKey=name
100+
// +patchStrategy=merge
101+
Env []corev1.EnvVar `json:"extraEnv,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,11,rep,name=extraEnv"`
102+
// PodTemplateSpec describes the data a pod should have when created from a template
103+
// +optional
104+
PodTemplate *corev1.PodTemplateSpec `json:"podTemplate" protobuf:"bytes,12,opt,name=podTemplate"`
81105
}
82106

83107
type ConditionType string

0 commit comments

Comments
 (0)