ios – WebRTC: While silence detected, The timestamps getting slower and the incoming stream stops

I have noticed an issue while using WebRTC in my iOS mobile app. I aim to record both outgoing and incoming audio during a call, and I have successfully implemented this feature.

However, I am encountering a problem when the speaker doesn’t hear anything, either because the other user has muted themselves or there is just silence. During these silent periods, the recording stops.

For instance, if there is a 20-second call and the other user mutes themselves for the last 10 seconds, I only receive a 10-second recording.

Could you please provide guidance on how to ensure the recording continues even during periods of silence or when the other user is on mute?

What steps will reproduce the problem?

In neteq_impl.cc, I added a listener in decodeLoop:

strcpy(filepath, getenv("HOME"));
strcat(filepath, "/Documents/MyCallRecords/inputIncomingSide.raw");
const void* ptr = &decoded_buffer_[0];

FILE* fp = fopen(filepath, "a");
size_t written = fwrite(ptr, sizeof(int16_t), decoded_buffer_length_ - *decoded_length, fp);
fflush(fp);
fclose(fp);

And in audio_encoder.cc:

const void* ptr = &audio[0];
char buffer[256];
strcpy(buffer, getenv("HOME"));
strcat(buffer, "/Documents/MyCallRecords/inputCurrentUserSide.raw");    
FILE* fp = fopen(buffer, "a");
if (fp == nullptr) {
    AudioEncoder::EncodedInfo info;
    info.encoded_bytes = 0;
    info.encoded_timestamp = 0;
    return info;
}
size_t written = fwrite(ptr, sizeof(const int16_t), audio.size(), fp);    
fflush(fp);
fclose(fp);

What is the expected result?

The expected result is that the timestamps of the frames will keep going even if it hears silence or the user is muted.

What do you see instead?

The timestamps move slower than expected, and the recording stops.

Additional Information:

Both incoming and outgoing audio are using a 48kHz sample rate.
Frame size differences:

  • Incoming audio is processed in frames of 2880 samples (60 milliseconds at 48kHz).
  • Outgoing audio is processed in frames of 480 samples (10 milliseconds at 48kHz).

Processing frequency:

  • Incoming audio logs every 63-64 milliseconds.
  • Outgoing audio logs every 20-22 milliseconds.

Buffer management:

  • Incoming audio has a buffer length of 5760 samples but only 2880 are processed each time.
  • Outgoing audio processes all 480 samples in its buffer each time.
    Timing consistency:
  • Incoming audio shows very consistent timing between logs.
  • Outgoing audio shows slight variations in timing between logs.

What version of the product are you using?

WebRTC commit/Release: 6261i

On what operating system?

OS: iOS Mobile App
Version: Doesn’t matter

Thank you.

Read more here: Source link