WebRTC is the Problem | Lobsters

A lot of this needs an asterisk: “.. for the default existing implementations of WebRTC”. Jitter buffer sizes and time-stamping etc can all be completely modified, especially if you control both ends of the pipe.

It may however require forking the WebRTC libraries. I’ve found that webrts-rs has some.. questionable practices within the implementation and doesn’t expose all the levers you would need for a good application on top.

But the core of WebRTC (ie the protocols and general ideas) I find pretty solid and honestly quite flexible. For instance we use it to receive video streams for archival and have configured an exceptionally long buffering time on the server and a custom time-stamping system on the client.

But I agree with the article overall. It’s so obviously designed for video conferencing that the more you depart from it the more painful it becomes. I don’t think there’s a good reason to use it for voice in open ai. This is a simple, targeted thing that doesn’t need the more advanced features sets.