Skip to content

feat: refactors, added new extraEnvVar, extraEnvFrom, runtimeClassName and podTemplate for Model CRD, fixes #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nekomeowww merged 1 commit into from
Nov 9, 2024
Merged

feat: refactors, added new extraEnvVar, extraEnvFrom, runtimeClassName and podTemplate for Model CRD, fixes #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 2 additions & 28 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,34 +145,8 @@ Unlock the abilities to run the following models with the Ollama Operator over K

> [!TIP]
> By the power of [`Modelfile`](https://github.com/ollama/ollama/blob/main/docs/modelfile.md) backed by Ollama, you can create and bundle any of your own model. **As long as it's a GGUF formatted model.**

| Model | Parameters | Size | Model image | Full model image URL | Multi-modal | Uncensored |
| ------------------------------------------------------------------ | ---------- | ----- | ------------------- | ---------------------------------------------- | ----------- | ---------- |
| [Phi-3 Mini](https://ollama.com/library/phi3) | 3.8B | 2.3GB | `phi3` | `registry.ollama.ai/library/phi3` | | |
| [Llama 3](https://ollama.com/library/llama3) | 8B | 4.7GB | `llama3` | `registry.ollama.ai/library/llama3` | | |
| [Dolphin Llama 3](https://ollama.com/library/dolphin-llama3) | 8B | 4.7GB | `dolphin-llama3` | `registry.ollama.ai/dolphin-llama3` | | ✅ |
| [WizardLM-2](https://ollama.com/library/wizardlm2) | 7B | 4.1GB | `wizardlm2` | `registry.ollama.ai/library/wizardlm2` | | |
| [Llama 2](https://ollama.com/library/llama2) | 7B | 3.8GB | `llama2` | `registry.ollama.ai/library/llama2` | | |
| [Mistral](https://ollama.com/library/mistral) | 7B | 4.1GB | `mistral` | `registry.ollama.ai/library/mistral` | | |
| [Mixtral 8x7B](https://ollama.com/library/mixtral:8x7b) | 8x7B | 26GB | `mixtral:8x7b` | `registry.ollama.ai/library/mixtral:8x7b` | | |
| [Mixtral 8x22B](https://ollama.com/library/mixtral:8x22b) | 8x22B | 80GB | `mixtral:8x22b` | `registry.ollama.ai/library/mixtral:8x22b` | | |
| [Command R](https://ollama.com/library/command-r) | 35B | 20GB | `command-r` | `registry.ollama.ai/library/command-r` | | |
| [Command R Plus](https://ollama.com/library/command-r-plus) | 104B | 59GB | `command-r-plus` | `registry.ollama.ai/library/command-r-plus` | | |
| [Dolphin Phi](https://ollama.com/library/dolphin-phi) | 2.7B | 1.6GB | `dolphin-phi` | `registry.ollama.ai/library/dolphin-phi` | | ✅ |
| [Phi-2](https://ollama.com/library/phi) | 2.7B | 1.7GB | `phi` | `registry.ollama.ai/library/phi` | | |
| [Neural Chat](https://ollama.com/library/neural-chat) | 7B | 4.1GB | `neural-chat` | `registry.ollama.ai/library/neural-chat` | | |
| [Starling](https://ollama.com/library/starling-lm) | 7B | 4.1GB | `starling-lm` | `registry.ollama.ai/library/starling-lm` | | |
| [Code Llama](https://ollama.com/library/codellama) | 7B | 3.8GB | `codellama` | `registry.ollama.ai/library/codellama` | | |
| [Llama 2 Uncensored](https://ollama.com/library/llama2-uncensored) | 7B | 3.8GB | `llama2-uncensored` | `registry.ollama.ai/library/llama2-uncensored` | | ✅ |
| [Llama 2 13B](https://ollama.com/library/llama2) | 13B | 7.3GB | `llama2:13b` | `registry.ollama.ai/library/llama2:13b` | | |
| [Llama 2 70B](https://ollama.com/library/llama2) | 70B | 39GB | `llama2:70b` | `registry.ollama.ai/library/llama2:70b` | | |
| Orca Mini | 3B | 1.9GB | `orca-mini` | `registry.ollama.ai/library/orca-mini` | | |
| Vicuna | 7B | 3.8GB | `vicuna` | `registry.ollama.ai/library/vicuna` | | |
| LLaVA | 7B | 4.5GB | `llava` | `registry.ollama.ai/library/llava` | ✅ | |
| Gemma 2B | 2B | 1.4GB | `gemma:2b` | `registry.ollama.ai/library/gemma:2b` | | |
| Gemma 7B | 7B | 4.8GB | `gemma:7b` | `registry.ollama.ai/library/gemma:7b` | | |

Full list of available images can be found at [Ollama Library](https://ollama.com/library).
>
> Full list of available images can be found at [Ollama Library](https://ollama.com/library).

> [!WARNING]
> You should have at least 8 GB of RAM available on your node to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Expand Down
24 changes: 24 additions & 0 deletions api/ollama/v1/model_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,30 @@ type ModelSpec struct {
// More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes
// +optional
PersistentVolume *ModelPersistentVolumeSpec `json:"persistentVolume,omitempty" protobuf:"bytes,8,opt,name=persistentVolume"`
// RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used
// to run this pod. If no RuntimeClass resource matches the named class, the pod will not be run.
// If unset or empty, the "legacy" RuntimeClass will be used, which is an implicit class with an
// empty definition that uses the default runtime handler.
// More info: https://git.k8s.io/enhancements/keps/sig-node/585-runtime-class
// +optional
RuntimeClassName *string `json:"runtimeClassName,omitempty" protobuf:"bytes,9,opt,name=runtimeClassName"`
// List of sources to populate environment variables in the container.
// The keys defined within a source must be a C_IDENTIFIER. All invalid keys
// will be reported as an event when the container is starting. When a key exists in multiple
// sources, the value associated with the last source will take precedence.
// Values defined by an Env with a duplicate key will take precedence.
// Cannot be updated.
// +optional
ExtraEnvFrom []corev1.EnvFromSource `json:"extraEnvFrom,omitempty" protobuf:"bytes,10,rep,name=extraEnvFrom"`
// List of environment variables to set in the container.
// Cannot be updated.
// +optional
// +patchMergeKey=name
// +patchStrategy=merge
Env []corev1.EnvVar `json:"extraEnv,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,11,rep,name=extraEnv"`
// PodTemplateSpec describes the data a pod should have when created from a template
// +optional
PodTemplate *corev1.PodTemplateSpec `json:"podTemplate" protobuf:"bytes,12,opt,name=podTemplate"`
}

type ConditionType string
Expand Down
Loading