Pulseaudio sound delay in firefox

Every now and again, I will get a 1-2second sound delay from audio played in Firefox (eg. Youtube or Twitch); when I reload the corresponding tab (re-registering the stream), the delay is gone. This happens in one of the two cases, the first of which I can consistently reproduce:

I change the input configuration of my AV receiver to the correct port -> the receiver takes some time until that’s done, and I’m guessing pulseaudio expects it to be done immediately -> this introduces a delay for all open tabs
Sometimes, an open tab will suddenly be affected without any changes in audio configuration – if the sound is playing, it will get stuck for a second and then continue playing where it left off (but the video continues running, so now they’re out of sync, and sound also continues to play for a second after I stop the video)

I have tried to reproduce this for other audio sources, but it only happened in Firefox (eg. I opened a video in Chrome and one in Firefox, did the input config switching, and only Firefox got a delay). Also, the delay seems to happen before the sound reaches pulse, since when I open pavucontrol, I see the volume meter with a delay as well.

One detail that might be relevant is that I’m running over a combined sink (load-module module-combine-sink) that includes the AV receiver with the corresponding configuration mentioned before as one of its outputs, so one of these sinks (dis)appears when the configuration is switched from/to the correct one. (see config below)

Has anyone else encountered this problem, and is there maybe a fix for this that isn’t just reloading tabs left and right? Google didn’t get me far on this…

Relevant config:

set-card-profile alsa_card.pci-0000_08_00.1 output:hdmi-surround-extra1 # output config
# create sink that streams to my receiver on the specified port:
load-module module-combine-sink sink_name=combined slaves=...,alsa_output.pci-0000_08_00.1.hdmi-surround-extra1
set-default-sink combined

Versions:
openSUSE Leap 15.4
Firefox 115.2.1esr
Pulseaudio 15.0