Multimodal support in IGW (initial triage)

[rahulgurnani]

Presently, IGW doesn't support multimodal completely. The following are the gaps:

• Support for video/audio data formats
• Support for prefix caching. The present strategy may not work based on discussion in https://www.youtube.com/watch?v=-WqJ8PbnFg8 . This will be addressed in issue MultiModal approximate prefix cache aware #2172

For further investigation, refer: https://docs.google.com/document/d/1YKx31zmF-mMGgb41InQfNmaOIVlAElsUXHIgHfQe_68/edit?tab=t.0

[rahulgurnani]

/assign

[ahg-g]

Can we limit the scope of this issue to "framework" support, specifically request parsing.

[rahulgurnani]

Can we limit the scope of this issue to "framework" support, specifically request parsing.

Sure, updated the issue description.

[pierDipi]

is transcription endpoint support in scope for this issue?

[rahulgurnani]

/milestone Multimodal Model support

[rahulgurnani]

Scoping this issue to support for video/audio formats. Other tasks can be addressed in the separate issues.