Feature request: Controlling `config.json` of HF's models

[keyboardAnt]

Thanks for this great project!

A quick feature request: When loading models from the HuggingFace Hub, allow providing custom values to overwrite the default config.json. For example, change the value of max_position_embeddings from 32768 to 100000, to allow longer prompts.

[kausky]

hey, any workarounds or development on this?

[AkshatGiri]

Also would be super useful for models that don't provide a config.json.

[kausky]

Being worked on in #2547