Embedding (sets of) suggested sampling parameters in GGUF files

[dagbdagb]

I hope this is the right place for this, seeing how this repo holds the file gguf.md. Assuming this is the 'official' spec for the file.

As I understand it, whoever makes a model often suggests the optimal sampling parameters for their model. Which strikes me as (slightly) odd, given that the optimal set may depend a great deal on the primary task at hand. Coding vs creative writing, for example. AFAIUI.

My idea/question is rather simple: would it make sense to allow for embedding sets of sampling parameters in the GGUF file? This allows both the org behind the model as well as people offering quants to add sets of sampling parameters for the model and task in question.

And users can then fire up llama.cpp without having to figure out the general 'good' sampling parameters for the model in question, because llama.cpp can read that from the GGUF file directly.

With the option of letting users choose a set (suggested by the maker or quantizer) better aligned with the purpose in question, or override the suggested/built-in settings by using the existing mechanism (in llama.cpp) for applying sampling parameters, of course.

Unsure if having a standard list of purposes is wise or not. general, coding, creative writing, etc...

[Green-Sky]

see ggml-org/llama.cpp#17120

[dagbdagb]

thank you! Seeing how gguf.md did not mention this (as far as I could see), I took for given it was missing...