Mismatched filter and arbitrary waveform radar range resolution

However, how can one compute or at least define range resolution limits for an arbitrary mismatched filter or an arbitrary waveform (e.g. a Barker code) ?

You use a stochastic model for your noise, add that, find an ML estimator for range, or use your own estimator and describe it in stochastic terms, then find the standard deviation of range as estimated under that noise.

So, in the end, it’s “just” transforming random variables and keeping track of variance.

Empirically, you look at the ambiguity function of the convolution of your transmit and your receive system, and find the marginal distribution (of only range).

Other than that, neither the range resolution nor the sidelobes have identified limits, and thus are arbitrarily good or bad depending on what (filter, transmitted signal) one picks.

Sure. I mean, if you pick a receive filter that has nulls exactly where your signal has energy, then your estimator can estimate nothing.

This seems supported by old and new papers creating mismatched filters

This isn’t that surprising. The one thing that a matched filter optimizes in AWGN is the SNR. It doesn’t optimize for range certainty.

In effect, you do want the output of the “transmit pulse – pulse shaper – channel – receive filter” system to be as “pulsey” as possible, so the intuitive approach would look more like an equalizer than a matched filter, if you asked me.