[IcM] Speech via Cognitive Services does not work on Cordova on Android

[compulim]

Screenshots

(TBD)

Version

4.9.2

Describe the bug

Customer reported Web Chat hosted on a Cordova app on Android with speech enabled through Cognitive Services does not work.

From the support ticket:

Customer has integrated bot into angular application and text-speech and speech-text functionality works fine. He has converted the same application into Cordova app where clicking on mic, it doesn’t work. Microphone doesn’t even listen to the audio.

It seems speech recognition is working but not speech synthesis, need to investigate.

Steps to reproduce

From the support ticket:

• Set up android studio and collected HAR trace using fiddler as proxy.
• We have the angular source code and the .apk file.
• Created new bot and speech service and replaced the secrets in Customer’s code.
• I have also made changes to the code so that it requires permission to Microphone and provided access to microphone. Still it fails.
• In fiddler, I see that to Speech service, after GET /speech/recognition/conversation/cognitiveservices/v1, I don’t see any request to POST /cognitiveservices/v1. Is it a problem because the Microphone not recording the audio input?

Expected behavior

Speech via Cognitive Services should work on Cordova on Android. If not working, we need to understand why and document it in docs/SPEECH.md

Next steps

• Repro the issue with a vanilla Cordova app with appropriate app permissions

Additional context

The ticket number is 194036845/120060926003891.

[Bug]

[compulim]

Tested with Cordova 10 on an Android 10 device.

A permission dialog is required to enable speech support inside Cordova app.

I tested this Cordova plug-in cordova-plugin-k-webrtc-permission and it can correctly request permission and enable speech support in Web Chat.

The plug-in request for permissions more than Web Chat is needed, so I would suggested looking for another approach similar to what this plug-in does.

[compulim]

We need to add more documentation before closing this issue.

[compulim]

Moving to R11 for documentation update.