Need Guidance for Real-Time Output in Orpheus TTS

[mishradibyajyoti]

Hello,
We are using Orpheus TTS and have tested it on an NVIDIA H200 server. For a 10-word input, we are getting around 1.9 seconds latency.
Could you please guide us on how to achieve real-time or near–real-time output?
Any best practices, recommended configurations, or streaming support would be very helpful.

[jjmlovesgit]

The FastAPI set up can deliver fast responses:

Please consider this thread: #221

Take the code from the repo referenced and share with ChatGPT and add your code and ask it analyze and plan for how to merge the two etc. Vibe code your way to using this. ChatGPT can analyze my code explain it and then give you working code to iterate on with the fastAPI set up.