In various parts of the world, firing into the air is a very standard gesture of high emotions at celebrations or at funerals. The West maintains this as a traditional part of military funerals.
At military funerals, however, the guns are loaded with blanks and this is purely for display, because we don't want to create more funerals! Elsewhere though, this does kill people. The link above provides a long list of cases; most notable in the US are probably Armando Martinez, a member of the Texas House of Representatives, and Shannon Smith, whose death prompted Arizona to make this illegal.
So why don't shotgun pellets injure people? Simple answer: they do. As Dick Cheney knows, accidents happen when you aren't very careful about where you're firing and who's over there!
This gives you the answer why skeet shooting doesn't injure people with falling shot. The location and direction of fire are carefully chosen so that nothing important is in the arc of fire, within the range of a shotgun pellet. If this is in an area which is likely to have members of the public around, for example on farmland, access points to the area should be blocked and red-flagged. The operator for the skeet thrower is situated so they are out of the line of fire too. On pheasant shoots where the birds are of course unpredictable, each person is assigned a "peg" and told where they are allowed to fire, precisely to prevent falling-shot injuries.
It's worth noting though that all this still relies on having good discipline with a loaded weapon. Accidents do still happen when skeet shooting.
While the lower terminal velocity of shotgun pellets may make them "less dangerous" than other projectiles, this does not make them safe by any means. Using dimensions from Wikipedia and a terminal velocity calculator, and assuming a Cd of 0.1 for pellets (the standard figure for a smooth round ball), the terminal velocity of a BB pellet is 162mph, which is about the same as the fastest tennis serve ever. Even for #10 shot, terminal velocity is 103mph, which is around the world record speed for a baseball pitch. Also for reference, the muzzle velocity of a BB gun is generally above 60m/s, or 134mph. Based on these numbers, the risk to life from falling birdshot is roughly equivalent to placing a BB gun against your head and pulling the trigger.
For completeness, I will note that @user560822 says they have obtained terminal velocity figures from the Chairgun app which are roughly half my values. I don't have access to this app, or details of how the app does its calculations. Airgun pellets are not smooth round balls, and I've seen references to bullets having Cd of 0.3, so maybe this would affect the figures from Chairgun and make them less applicable to figures for a shotgun pellet; or of course maybe my figures are based on an incorrect assumption. I'll allow that my numbers above may be an upper value, but I can't easily find better data to make this more accurate.