Why don't we percieve chords like we perceive the mix of two light waves?

Question

Why is the combination of two light waves (red, yellow) percieved as the same color as the arithmetic mean of their frequencies (orange) while we percieve two musical notes at the same time as just those two waves stacked on top, and not the mean of those frequencies?

Answer 1

Because 16,000 is greater than 3.

We only have 3 sorts of detector (called cones) in the eye, sensitive broadly to red, green and blue light. So a mix of red and green light excites the red and green cones. But yellow light, in between red and green, also excites the red and green cones, and the brain can't tell the difference. So it's not the mean of the frequencies - a red-blue mix is different from green - but there is some averaging going on.

But the ear has 16,000 hair cells each sensitive to a particular frequency, so the brain gets a whole lot more information. A 256 Hz C will excite the 256 Hz hair cell, but a mix of 242 Hz B and 271 Hz C# will excite those two receptors and not the one in between.

Answer 2

Why is the combination of two light waves (red, yellow) percieved as the same color as the arithmetic mean of their frequencies (orange)

Simple: no, it isn't. This is not how color perception works. (If you're inclined to disagree, ask yourself: how does your model account for the fact that changing the relative intensity of the two light sources changes the perceived color?)

But in any case, there is no a priori reason why the human eye and the human ear (and the associated psychophysical processes for color and sound perception) should work the same way.

Answer 3

Colour perception isn't as simple as you think

Roger Barlow has already described why sound perception is different (you have enough receptors to distinguish bot a wide variety of notes and their strength at the same time).And the eye has only 4 receptor types.

But human colour perception is also different for another important reason. It is not a matter of creating the perception of the mean frequency. It is also possible to create colour perceptions that do not exist as single colours in any spectrum.

The reason is that the sensitivity of colour receptors are broad and overlapping. Making sense of that in the brain isn't as simple as "averaging" the signals. Also, we would overload our optical nerve capacity if we tried to transmit a large signal containing data from all the pixels so a lot of pre-processing is done before the signal gets to the brain. But the overall process for colour is far more complex than averaging.

On reason is that the signals from the receptors are pre-processed to give pairwise differences between receptor signals (especially the red-green pair and the difference between the red+green and the blue). This is one reason for common colour blindness as, if the red-green receptors overlap too much, it bugger the ability to distinguish the two colours (completely missing receptors is a far less common cause). The brain then synthesises perceived colours from these difference signals.

A consequence of the complex processing is the ability to see non-spectral colours like magenta: a colour that never appears in a single wavelength spectrum. The brain doesn't average a signal containing red and violet light (that would average to green): it synthesises a new colour. This shows the brain is creating colour perception is a more complex way than "averaging" would suggest.

In short, colour perception is necessarily different to sound perception. Colour is a complex thing synthesised from just 4 signals (roughly brightness + the three colours though brightness only matters much in low light). there are thousands of sound receptors and they can all send signals at once so we can perceive many simultaneous sounds (with little spatial detail) but only one perceptual colour (for each region and with far more spatial detail).

Answer 4

The answer to any human perception question is "its complicated."

We evolved for maximum fitness (for some fitness function). The way we hear sound and the way we hear light are the most effective ways we have found to deal with the signals.

If we try to read into it further, however, its worth noting that its useful to be able to identify "individual" sources of sound. You can do a lot once you have individual sounds isolated from each other. Virtually all sounds we hear in the natural world have a harmonic character to them. You never hear a 440Hz sine wave in the forest. You always hear 440Hz mixed with others (such as 880Hz and 1760Hz), and the relative amplitudes of these frequencies are reasonably fixed because they're defined by physics.

There's little advantage in being able to hear each frequency and then to do some heavy duty processing deeper in the brain. Instead what we have found it its most valuable to take sound and basically split it up into a fundamental frequency and a sound "color" which captures the associated overtone amplitudes. Its this color which is why a trumpet sounds different than a flute, even if they play the same note.

Indeed, if you split up the signal in this way, you can do some remarkably processing. Our ears "color" the sound based on direction, emphasizing different frequencies depending on which direction the sound approaches the ears. If our brains can identify the "color" of the sound, we can actually back out this frequency-dependent response and figure out the direction of any given signal (even if there's a bunch of other sounds going on).

For a fun aside, there's a trick in the electronic music community where you leave off the fundamental on your low bass notes. Leave it off completely, and only have the harmonics above. Our brain will recognize the "color," and fill in the fundamental for you. This is great because it can take a lot of subwoofer power to reproduce the fundamental, and quite a lot less power to reproduce the harmonics. If the musician can rely on the human brain to fill it in, their music will sound better on weaker sound systems!