We support hosting models in two ways:
- APIs
- Bulk jobs
on the following systems:
- Kubernetes
- Openstack
Current direction of the project is to be the open source alternative to NVIDIA NIM and have support for multiple architectures and inference engines.
Read the TLDR usage guide.
There are blog articles for each category of task accompanied with youtube videos and over 300 examples for text, vision and audio models.
- Website: geniusrise.ai
- Docs: docs.geniusrise.ai
- Examples: geniusrise/examples
- Cloud: geniusrise.com
- pytorch
- transformers
- peft
- accelerate
- DeepSpeed
- bitsandbytes
- AutoAWQ
- AutoGPTQ
- flash-attention
- vllm
- llama-cpp-python
- llama.cpp
- whispercpp
- whisper.cpp
- faster-whisper
The entire project is Apache 2.0 licensed.
Take a look at good first issues or help wanted on the board.
Or feel free to contact at [email protected] for more.
