How do we choose between two scientific ideas when both aren't yet falsified?

Question

Consider a leading theory for a phenomenon, then how are new theories shown to be better while both remain unfalsified?

Answer 1

The very bare minimum that a theory has to obey to be considered as viable is:

• The theory is self-consistent (it cannot predict two different results for the same input)
• The theory is complete (within its domain of applicability, any statement can be decided by the theory)
• The theory agrees with all past experiments

The falsifiability part is basically related to the third criteria: if a newer experiment cannot be explained by the theory, it is falsified.

What to do if two theories on the same domain of applicability obey all three? Here are a few options!

Don't do anything!

For a while now, physics has overall tried to be about applying models more than trying to find some kind of metaphysical truth. With this in mind, there is nothing fundamentally wrong about having two theories for the same domain, especially if they have about the same predictions for any reasonable experiment.

Things may not be ideal, of course. Competing theories may have wildly different predictions (although that is less of a problem in physics usually). Also there are finite resources for research, and it may be useful to determine which theory is more promising, more practical, or closer to the fundamental truth. There are many, many criteria I could bring up, stemming from made-up examples (it's a big hobby of philosophers of science), but I'll try to stick to realistic cases.

Apply heuristics

There are many broad bias in physics which have shown to be more likely to lead to better models than others. There are no fundamental reasons why these should lead to better theories, but so far they seem to be productive, if not metaphysically true :

• Try to go for the simpler theory. "Simpler" is of course somewhat vague, and different people may have different opinions on what constitutes a simpler theory. A few cases can be useful, though. Overall, people will pick general relativity over the Einstein-Cartan theory (general relativity + a torsion field), Brans-Dicke theory (general relativity + a scalar field), or other versions that are general relativity with additional terms, since any of those additional objects have such low impact on actual experiments. We went with quantum chromodynamics rather than the patchwork of formulas we had with the meson zoo of particles.
• Try to go for a local theory. It's generally assumed that causal effects are always local, and only propagate from their point of origin.
• Symmetries tend to play a big role in modern theories, and relatedly, conserved quantities that stem from them. We tend to favor theories if they stem from some underlying principle of symmetry, and we prefer if the big conservation laws hold, at least locally (conservation of energy, momentum, charge, etc). Fred Hoyle's gravitational theory was in part not liked too much for breaking local time translation symmetry, and therefore conservation of energy. The neutrino was inferred from conservation of energy, and the theories that went against it didn't pan out.
• The second law of thermodynamics looms large in a lot of physicists' minds, and overall, any theory that would imply a break of it is considered suspicious.
• Causality is generally considered very important. This is why naked singularities, closed timelike curves or time-symmetric theories are not considered to be likely to be physical. This is linked to another concept, determinism. While not all modern theories are deterministic, they are at least probabilistic. Theories that allow for events to happen with no assigned probability are generally considered pathological.

Beware that many such heuristics existed in the past and have been shown to be wrong, or at least not necessarily best. Galilean invariance, celestial objects moving on circles, etc.

Go for the practical theory

A big part of the appeal of a theory is its ability to get results, and get them easily enough. If a theory seems promising, but the mathematics necessary to even obtain measurable outcomes isn't there, or fairly intractable, it may be best to go with the simpler theory. It's best to keep measurable quantities obtainable, in realistic scenarios, with reasonable computation times.

Another practical aspect: ideally, a theory should be such that we can apply it without omniscience. If a theory requires the knowledge of too many initial conditions, it is not going to be terribly applicable in experiments. Ideally the influence of most of the universe should be either very small, or can be approximated by some general principles. This is common in most theories: inverse square laws, the cluster decomposition principle, mean fields, etc.

Raw philosophical bias

A lot of choices in theory on the individual level is simply due to their own ideas on ontology, causality, epistemology, and other metaphysical business. This is what drives a lot of adherents of fringe theories, the fights between competing theories for new physics, as well as some of the choices regarding what is established science. It's entirely possible to have theories without any ontology, where we simply input a list of measurements in formulas and get another list of measurements, but in addition to being impractical, most people believe in some kind of physical objects, and the theories tend to reflect those ontologies. A lot of people don't like the Copenhagen interpretation due to its lack of determinism, or its contextuality. Many of the heuristics I listed are rooted in philosophical positions, in addition to being seen as generally useful. There was some resistance to general relativity from the idea of Kant that Euclidian geometry was fundamental. A lot of science in the Soviet Union was constrained by the requirement that it had to obey dialectic materialism, which favored deterministic quantum theories and eternal cosmologies. Opinions on what constitutes a reasonable spacetime in general relativity can be rooted a lot in the author's opinion on causality or the nature of time (presentism, growing block, eternalism, whether time is cyclical, eternal or has a starting point, etc).

Answer 2

Some general philosophy-of-science answers...

• Occam's razor - prefer the "simpler" theory (of course a lot to discuss here)
• Generality - prefer the theory that explains more phenomena
• Specificity - prefer the theory that makes more detailed predictions (e.g. quantitative rather than just qualitative)
• Falsifiability - prefer the theory that is easier to falsify

A related, but personal, thought: there are downsides of viewing science as a search for the One Ultimate True Theory. There are infinitely many theories that cannot be distinguished by observations we can make in this universe. If one of them is "true", all of them are. So there is no One True Theory. So there's no mandate to always be picking between competing theories; this could be a fool's errand. Instead maybe science is about something else, say, a search for the most accurate (counterfactual) predictions: trying to be able to say what would happen if. In that case, theories would just be a means to this end and picking between them becomes more pragmatic and hinges a lot on falsifiability.

Answer 3

In this case they are not theories, but hypotheses. You use the one that seems more plausible or better explains the context that you work with.

In practice there are many non-scientific considerations that come into play: different schools/communities adhere to different hypotheses (particularly when the hypothesis originates form your supervisor or the supervisor of your supervisor). Sometimes they fight bitterly - both in journal pages and conferences, but also when it comes to getting positions, grants, etc. So one spends many years working in the framework of a particular hypothesis in a hope that it will be proven correct in the end.

Answer 4

Take quantum physics as a good example: regarding it as the "better" theory of everything does not mean that classical models need to (or will be) falsified. They will continue to be valid, and explain observations of our everyday lives with sufficient accuracy. What we want, is to see the building blocks of the old model emerge from what we get when we no longer seek to replace parts from old theories by new ones, but apply the new ones exclusively.