I have a 12 V power supply that connects into a custom made power board that then has three 5 V outputs. There are three things connected to it's working all the time.
How does that work, because 3 x 5 is 15 not 12?
I have a 12 V power supply that connects into a custom made power board that then has three 5 V outputs. There are three things connected to it's working all the time.
How does that work, because 3 x 5 is 15 not 12?
That is a power supply, so its job is to generate the voltages you need.
First observation is: A voltage is a voltage, and there's no inherent reason you should be thinking they need to add up.
For example, the three 5V outputs could simply be independent – meaning that the power supply actually looks more like:

simulate this circuit – Schematic created using CircuitLab
As you can see, nothing here adds up voltages – so there's not even any question of how the output voltages' sum can be larger than the input voltage.
It's also quite likely that the negative supply sides are internally connected, so that out1- to out3- are actually internally connected. That still doesn't introduce any reason to think about the sum of these voltages – the voltage sources are in parallel, not series.
As a side note, even if the power supply actually generated 15 V from the 12 V input, that's not a surprise – there's step up converters for both DC to DC as well as from AC to AC (a typical example of the latter would be a simple transformer with more windings on the output side than on the input side).
It might be of interest to you to learn a bit about basic linear networks, and when to add voltages or currents and when not to.

simulate this circuit – Schematic created using CircuitLab
Figure 1. 12 V input. 3 x 5 V output.
The circuit of Figure 1 shows the likely internal arrangement.
Your view that voltages have to add up is flawed. A similar add-up flaw applies to currents. But when you consider power, the add-up rule does apply. That is, real power entering is always equal to or larger than real power delivered. Any missing power (where power out is less than power in) is dissipated as heat.
For your example, you'd measure power delivered to each of those three devices (current1 x 5V + current2 x 5V + current3 x 5V). Then compare with power into your power board (current in x 12V).