New Model and Cross-region inference feature support

[xufanglin]

Customer need new model Haiku 3.5, and cross-region inference support under NO PG mode. Is there a roadmap?

[cloudbeer]

There are two ways for cross-region reasoning:

• Use the modelID for cross-region, and this modelID is only supported in certain regions. If you want to support it in no PG mode across all regions, then a lot of hard-coded combinations would be needed.
• The application performs load balancing, but this depends on a custom model, so a database is required.

Hardcode approach:
Modify the code: Modify the getModelDataWithoutDB method in /src/util/helper.ts; modify /src/model_data.ts, and add the configurations in these two files.

Use Performance mode, which can maintain concurrency performance while allowing customization of the model.