Many times in physics when we analyze a physical system mathematicly we get divergences, but when those divergences has no dependence on any actual physical quantity of interest we tend to disregard those "infinity Constants".
An example where such a thing happens is in the derivation of the Casimir force for two metal plates we get $\zeta(-3)$ which contains infinities.
One can provide many more examples where we make this analytical approach.
My question is why those infinities are treated this way, is there a deep mathematical reason why we do it?