Azure speech service continuous speech recognition

I’m pretty new to Azure speech service and I’m using twilo/plivo service for connecting a number with azure stt and process it further after transcription.

My problem is when I speak something, it’s detecting well, and when I stop speaking or stay silent, it will automatically process the empty speech which contains empty transcription text and it is returning it, this happens for every 10-15 seconds.. it automatically detecting speech.. I’m not cancelling the continuous recognition until the end of the call.

Anyone has similar experiences or anything I can change in speech configuration? Please let me know.

I used azure SDK and used both intial and speech segmentation timeout, but no change.. I’m using it for real time so I cannot add more then a second.