amdshark-ai: amdshark Modeling and Serving Libraries

amdshark Users

If you're looking to use amdshark check out our User Guide. For developers continue to read on.

Sub-projects

shortfin/

The shortfin sub-project is amdshark's high performance inference library and serving engine.

• API documentation for shortfin is available on readthedocs.

amdsharktank/

The amdshark Tank sub-project contains a collection of model recipes and conversion tools to produce inference-optimized programs.

• See the amdshark Tank Programming Guide for information about core concepts, the development model, dataset management, and more.
• See Direct Quantization with amdshark Tank for information about quantization support.

amdsharktuner/

The amdshark Tuner sub-project assists with tuning program performance by searching for optimal parameter configurations to use during model compilation. Check out the readme for more details.

Support matrix

Models

Model name	Model recipes	Serving apps	Guide
SDXL	amdsharktank/amdsharktank/models/punet/	shortfin/python/shortfin_apps/sd/	shortfin/python/shortfin_apps/sd/README.md
llama	amdsharktank/amdsharktank/models/llama/	shortfin/python/shortfin_apps/llm/	docs/shortfin/llm/user/llama_serving.md
Flux	amdsharktank/amdsharktank/models/flux/	shortfin/python/shortfin_apps/flux/	shortfin/python/shortfin_apps/flux/README.md

amdshark Developers

If you're looking to develop amdshark, check out our Developer Guide.