Switch between models during a conversation.

[m-mohsin-ali]

Hey how can i dynamically load and use models during a chat conversation for example on the bases on some classification or some specific trigger user text input can i lets say change my backend model from Blender 90m to wizards of Wikipedia model in-between the conversation without restarting the server. I am using /parlai/chat_service/tasks/chatbot/worlds.py as base example and parlai/chat_service/services/browser_chat as a testing platform.

[klshuster]

you can specify multiple models in the models key of your chat config; this will load the models you desire. You would then need custom logic in your worlds.py to handle the multiple models

[kidussolo]

How can i do that?

[klshuster]

i meant, under the override key, add as a key/value pair

[kidussolo]

It's still giving me the same error.

models:
    blender_90M:
      model: transformer/generator
      model_file: zoo:blender/blender_90M/model
      interactive_mode: True
      no_cuda: True
    wizard_of_wikipedia:
      model: projects:wizard_of_wikipedia:wizard_transformer_ranker
      model_file: zoo:wizard_of_wikipedia/full_dialogue_retrieval_model/model
      interactive_mode: True
      no_cuda: True
      override:
         model: projects:wizard_of_wikipedia:wizard_transformer_ranker
         model_file: zoo:wizard_of_wikipedia/full_dialogue_retrieval_model/model
         init_opt: /Users/kidussolomon/Downloads/agents/venv/lib/python3.9/site-packages/data/models/wizard_of_wikipedia/full_dialogue_retrieval_model/model.opt
         task: wizard_of_wikipedia:BasicdialogTeacher

[klshuster]

ok, you may need to explicitly add some of the model architecture keys (under the override key):

n_positions: 1000
ffn_size: 1200

[kidussolo]

Thanks this works.