Theorists (physicists) suggest that there is the term/entity, the Multiverse that contains a huge number of universes not necessarily like our own.
I personally find this theory very elegant because its explains the probabilistic outcomes of the experiments with fixed conditions on a quantum level.
It's obviously very hard to prove the validity of the theory of multiverse, but
Any particular theory of (astro)physics that predicts a multiverse will be disproven the moment experimental results or observations are found to be in conflict with the theory. In physics all theories are falsifiable, so this is not a problem in principle. However, you can then postulate that there may still exist a multiverse and that this idea cannot be proved wrong. But physics is about building falsifiable theories that can explain the physical world, it is not about defending or attacking the idea of multiverses.
A good example is the Many Worlds Interpretation of quantum mechanics (MWI). Obviously MWI is easily falsifiable, if you detect non-unitary time evolution in a well isolated system well within the predicted decoherence time scale, then you have disproven the MWI. A more interesting question is whether one can disprove collapse theories. As pointed out by David Deutsch, there is an experiment that you could in principle do to falsify the idea that the wavefunction undergoes a non-unitary collapse when performing a measurement.
This involves implementing an observer in a quantum computer and then doing a measurement that can have different outcomes. You then let the quantum computer evolve according to the unitary transform that reverses the act of performing the measurement but such that the observer will keep its mememory of having performed an observation. What then happens is that the quantum state of the measured system will have been restored to is original state while in collapse theories the final state will not be the same as the initial state.
I agree with you that the mathematical multiverse theory of Max Tegmark is the triumph of Occam's razor in simplicity (Although I disagree in restricting it only to Godel computable, mathematical structures). Not only it gets rid of explaining the physical universe itself (why there exist a physical universe?) but has many other philosophical advantages that I will not discuss here. Unfortunately for your answer, it is not a scientific but a metaphysical question, so you either believe it or not based on other benefits beyond it being provable. Regardless of the efforts of many physicists includind Max Tegmark, I do not think it could ever be proved or disproved. It is an unfalsifiable proposal (yes, you can try some restrictions to make some "predictions", but in the end you can always change it to make it unfalsifiable).
It is important to specify which multiverse theory one wants to disprove. Some are possible, others are not.
Tegmark's "Level I" model, where the universe is spatially infinite and hence contains regions mirroring our world (and other possible worlds) is falsifiable if we ever find evidence that the universe has a finite volume, for example by detecting a compact topology.
Other inflation domains (Level II) might hypothetically be disproven if we find evidence that chaotic inflation did not take place. The many worlds interpretation of quantum mechanics is in some sense true by definition since it is an interpretation of standard quantum mechanics, so to disprove it would require overturning quantum mechanics. Still, if there is actual wavefunction collapse and it can be observed, that would rule out the MWI - but the experiment looks very hard.
The really radical multiverse models like Tegmark Level IV, Meinong's Jungle or Schmidthuber's computable multiverse are trickier. For any conceivable experiment it seems that there will be worlds showing each possible outcome, so the experiment may not settle anything. Indeed, this problem is a reason many people reject such multiverse theories: there seems to be no way to gather evidence for or against them!
However, there might still be very peculiar methods to investigate them. One is quantum suicide: place yourself in a deadly situation. If you survive against all odds, either there is one universe and you were very lucky, or there is a multiverse and you only experience those worlds where your survival happened. If you repeat the process enough times you will have evidence that there is a multiverse - but only in some very rare branches, in all others your counterparts are dead and mourned (hence, don't do it). This is a weird example where you cannot falsify something, just verify it.
Much here depends on how one defines "probability of finding oneself in world X" or "probability of existing", which has some deep philosophical and mathematical difficulties in some of these theories (e.g. there are no probability spaces that work for Level IV, while there are for the computable multiverse). This also cuts to the core of many debates about the MWI: how do amplitudes turn into observable probabilities for observers in different branches?
