I do not think there is a particularly interesting relationship between the UV catastrophe and the Riemann-Lebesgue lemma.
The UV catastrophe is the observation that the spectral density of black body radiation diverges at high frequencies, when computed with classical physics. This is a problem, since the total power emitted by the black body is the integral over frequencies of the spectral density, which diverges. Quantum mechanics solves the UV catastrophe by introducing a "cost" to exciting high energy modes by proposing that a mode with frequency $\omega$ has a minimum energy excitation $\hbar \omega$, and the resulting spectral distribution goes to zero at infinite frequencies.
You can apply the Riemann-Lebesgue lemma to the correct, quantum-mechanical black body spectrum, known as the Planck distribution. Note that as physicists we would tend to call a move from the frequency to the time domain an inverse Fourier transform, but this is just a convention, and mathematically we can also think of this process as a Fourier transform, so we should be able to apply the Riemann-Lebesgue lemma.
The spectral density is an example of a power spectrum. By the Wiener-Khinchin theorem, the inverse Fourier transform of a power spectrum is an autocorrelation function. Then, the Riemann-Lebesgues lemma implies that since the Planck distribution is $L^1$ integrable (which it is since it gives a finite power), the autocorrelation function vanishes at infinity. I don't immediately see any particular physical significance to this statement. But, that would be the implication of the lemma applied to the black-body spectrum.
Perhaps you could try to use the converse of the Riemann-Lebesgues lemma to rephrase the original UV catastrophe in terms of the autocorrelation function instead of the spectral density. It's not obvious to me that this would work, though, since the Wiener-Kninchin theorem assumes the Fourier transform exists, which it may not if the spectral density diverges at infinity. Even if you could do this, I don't see why stating the UV catastrophe in terms of the autocorrelation function would give you any additional insight over the normal formulation in terms of the spectral density.