I am working on Fisher formalism and MCMC method. It seems that Fisher formalisme assumes that posterior is always Gaussian. So if I find with MCMC a gaussian posterior, I validate the results of Fisher computation (by getting a Gaussian distribution for both).
Now, I would like to know if the Gaussianity for Likelihood is always guaranteed, independently from the PDF I use.
More concretly, if I have the likelihood with a PDF $f(x)$ :
$$\mathcal{L} = \big(\prod_{k}\,f(x_{k})\big)\quad(1)$$
1) Under which conditions we have the Gaussianity ?
Maybe, one has to take the $\log$ of $\mathcal{L}$ and then write :
$$\log\,\mathcal{L} = \sum_{k}\,\log\,f(x_{k})\quad(2)$$
So we could conclude that from "Central Limit Theorem", the sum on $k$ of "$\log f(x_{k})$" follows, with a near factor of $N$ values $x_{k}$, a Gaussian distribution. (Actually, I think the average of random variables following the same PDF has a Gaussian $\mathcal{N}(0,1)$ distribution.
Is it systematically the case for equation $(2)$ ? I mean, this is the $\log$ which implies the sum and then the gaussianity with the "Central Limit Theorem" ?
So, this would be not the $\mathcal{L}$ which is gaussian (in equation $(1)$) but rather the $\log\,\mathcal{L}$ (in equation(2)).
From a rigorous point of view, I should write $f(x_k,\Theta)$ instead of $f(x_k)$ alone since $\Theta$ are the parameters of the model, shouldn't I ?
3) And Finally, if the gaussianity of $\log\,\mathcal{L}$ (or $\mathcal{L}$, I don't know for instant) is not always guaranteed, could anyone give me an example or an illustration where there is no gaussianity.
I hope to have been clear enough, feel free to ask me more precisions if need it.
Regards
ps: maybe this post should be moved to maths exchange forums or statistics forums.