java – Correct Complex Lens Distortion in FFMPEG (or OpenCV)

I need to process videos which were taken with a wide angle vehicle camera. A good example is this frame from a video:

As you can see, the image is pretty distorted and I need to correct that. After some research, neither a Barrel distortion or a Pincushion distortion is the case here. I’d say it depends on the axis. The x axis seems to have a Pincushion distortion, but the y axis seems to have a Barrel distortion.

In my project I use a Java backend and I use FFmpeg to process the videos (re-encoding, color correction, …). Therefore I’d like to do this in FFmpeg as another video filter.

I tried to use the lenscorrection filter but couldn’t find any suitable values for k1 and k2. IMHO this isn’t even possible since the filter can only handle the two types of distortion mentioned above. Here are two examples where I tried to correct the horizontal and vertical elements of the image:

(with k1= -0.7, k2=0.4)

(with k1=0.7)

But the resulting image would need to look like this (just roughly edited with GIMP):

Is this even possible with FFmpeg filters? I already tried out the v360 filter (similar like in this article) but just achieved similar results.

If I’m correct this isn’t possible with video filters. So I thought of doing this in OpenCV in Java by extracting all the frames with FFmpeg, transforming them with OpenCV and rendering them to a video again. Unfortunately I have no experience with OpenCV and couldn’t get any results. I think this should be possible in some way.

Can someone help me?