As per wikipedia, "brown" refers to long wavelength hues, yellow, orange, or red, in combination with low luminance or saturation.
So what about the short wavelength hues? Is there any reason why they are not called differently when desaturated?
Specifically, is there any physical reason? That is, is brown in some way qualitatively different from orange in some way, more so than desaturated blue from regular blue?
No, there isn't.
This is exclusively on the side of the perceptual, psychophysics side of colour perception. Colour is based on the spectral properties of light, but it is a complex quale that arises from a complicated combination of the biophysical response in the retina and the neural processing that takes the signals from the retina and transforms them into concepts that the brain-at-large can operate on. In particular, the naming of colours and the assignation of boundaries between them falls on the latter side of the equation, and this includes the bizarre fact that orange becomes a different colour when perceived as 'dim' (though note that this is context-dependent!) but cyan and yellow don't.
If you're interested in the subject, I'd recommend the Technology Connections video Brown; color is weird.
