Your observation is correct, though, as you might expect, it's usually phrased in more precise terms. The reason has to do with the Second Law of Thermodynamics. A good article on the subject is here (https://www.livescience.com/50941-second-law-thermodynamics.html), and I'll present an argument in my own words below.
The Second Law of Thermodynamics concerns the behavior of a quantity called entropy. Entropy essentially keeps track of the number of possible states a system can be in. You will often hear this phrased as "the amount of disorder in a system;" this definition works as long as you don't consider anything particularly crazy, but "amount of disorder" is not a quantitatively measurable thing, so it's not a particularly good formal definition. That said, either of those definitions should work to explain your observations, so feel free to use whichever one is intuitive for now.
With a little bit of applied statistics, it turns out that, in general, cold objects tend to be more ordered (i.e. they have less possible states they can be in). Hot objects, on the other hand, tend to be more disordered; since their molecules are moving faster, there are more states that the system has access to. If you assume that, on the microscopic level, every state of the system's molecules is basically equally probable (which is a good assumption for the vast majority of real systems), then you would probably conclude that eventually, a system, when left on its own, should be far more likely to end up in a configuration that covers a lot of possible states (which means it's in a configuration with higher entropy). Congratulations, you just derived the Second Law of Thermodynamics! Formally, it states that the entropy of a closed system does not decrease with time. Colloquially, it means that a system that is initially ordered (with, for example, separation between hot and cold) will tend toward disorder if left alone (where, for example, everything has warmed to about the same temperature). Creating a cold region is equivalent to creating a region of higher order, or lower entropy; such a process is statistically unlikely, so it's hard to do.
But there's another caveat we have to worry about. I was careful to state that the entropy of a closed system does not decrease with time; but it turns out that in order for entropy to stay exactly the same with time, you have to act on your system infinitely quickly, which is obviously impractical! So, for any real process taking a finite amount of time, the entropy of a closed system will increase. What does this mean, practically? It means that anything you do to try and, for example, create a cold spot, will inevitably have to waste some energy increasing the entropy of its environment. This wasted energy is precisely the heat that you observe.