Humans can detect sounds in a frequency range from about 20 Hz to 20 kHz. Then ultrasound is above 20 kHz
Occupational exposure to ultrasound in excess of 120 dB may lead to hearing loss.
How can ultrasound hurt human ears if it is above the audible range?
Ultrasound is used to break kidney stones and "burn" brain tumors. So no surprise that it can damage body parts, including inner ear. The effects of ultrasound on human body are multiple. One of them is heating due to absorption of the mechanical energy in the tissue and conversion to heat. This is used in physiotherapy as a desirable effect but it can be damaging if too much power is absorbed. There is also a mechanical effect. Parts reached by ultrasound vibrate so if the amplitude is too high they may break. The effect is used in ultrasound cleaners as well in medical field, to break apart stones and blood clots. Again, too much can do bad things. For this reason there are strict rules on maximum power allowed in any ultrasound imaging system. These effects have nothing to do with the fact that our ear can or cannot hear the sound. In similar way, your eyes can get burned by microwaves even though you don't see them.
Edit
Richard Tingle mentions an interesting aspect of this problem. I focused my answer above on the possibility of physical damage to the tissue due to ultrasound. In order to prevent these effectss, the limits for the maximum power used for ultrasound imaging is for example around 750 $mW/cm^2$ for sturdy organs like heart, liker, kidneys. For more sensitive areas like baby ultrasound or eyes the limits are much lower but stil of the order few (or tens of) $mW/cm^2$ . Now let's compare these with airborne ultrasound at 120dB level. In terms of intensity this means 1 $W/m^2$ or 0.1 $mW/cm^2$. And this is the same no matter if the sound is audible or not. The sensation we get depends on the frequency range but the physical intensity is the same.
This shows that the damage to the ears is of a different nature, and usually is a long term effect. One efect is an over-working of the sensitive hairs in the inner ear which in time can result of death of the cells in these hairs. The hairs are tuned to frequencies in the audible range. This does not mean that they do not vibrate when the frequency is off-resonance, just that the vibration amplitude is lower. Indeed, the limits established by the International Non-Ionizing Radiation Comitee for exposure to ultrasound are 110 dB for frequencies between 20 and 100 kHz (ultrasound) and just 75 dB for audible sound. This indicates that the effect of airborne ultrasound is less pronounced than that of sound. The difference between 110 dB and 75 dB is significant in terms of intensity. Hovewer the effect is here, as proved by many studies used by INRC to established their limits. The effects are not just possible hearing loss but also nausea, dizziness, headache and other things. So indeed 120 dB can be damaging even for ultrasound. You don't need to hear it to hurt you.
Sound, independent of frequency, is (or is caused by) periodic changes in local pressure. These periodic pressure changes can be turned into periodic movements, e.g. it‘s moving the eardrum and the other parts of the inner ear. Even when the sound can not be perceived by you (as sound), the eardrum and the other parts are still moved and they may be moved so strongly that they may get damaged.
An experiment to try (at your own risk):
Get a hold of one of those "cool mist" humidifiers -- the ones that use ultrasonic transducers, rather than just a wick and a fan. Fill the basin with water and turn it on. Put your finger over the transducer. It won't be long before you pull your finger away from the pain (personal experience).
Now imagine that happening to your ears.
Sound at particular high intensity (not frequency) can harm your hearing.
Even though ultrasound cannot be detected by humans ears, at high intensity (as for all sound) it can still cause damage to human ears. Ultrasound with intensity in excess of $120 dB$ can cause hearing damage.
If you have occupational exposure to 120 dB acoustic energy above 20 kHz, then either your ears are within a meter of, or in solid or liquid contact with the energy source. Through ambient atmosphere, ultrasound dissipates in much less distance than lower frequencies. At typical conditions, compared to 20 kHz, 100 kHz is attenuated 4 dB more per meter, i.e., its power drops a bit more than twice as quickly as you move away from the sound source.
What I feel that ultra sound and infrasound are not audible because infrasound generates small to and fro motion of our eardrum which is un observable and our ears cannot detect it (biologists should tell you the exact range)
Some reason can apply here. The to and fro motion is so fast that it becomes uninterpretable
Regarding the damage. It is like using a device with higher voltage and lower voltage than the rated voltage
Fast to and fro motion can result in damage of eardrums
