Is there any way to intuitively understand this ...
Let's look at the number of molecules in a given volume of air or gas inside the ball:
When one $cm^3$ of air outside the ball contains more nitrogen molecules than one $cm^3$ inside the ball, more molecules will have contact with the envelope of the ball outside than inside.
Therefore, more molecules will diffuse into the ball than molecules diffusing out of it.
Diffusion will stop when there is the same number of nitrogen molecules inside the ball as in the same volume of air outside. (Actually, there is still diffusion, but the same number of molecules diffusing into the ball is diffusing out of it.)
The same is true for $SF_6$ and all other gases, so in the end we will have air in the ball with the same composition and pressure as the surrounding air.
However, the probability that a $SF_6$ molecule touching the envelope diffuses is much lower than a nitrogen molecule. Maybe simply because the nitrogen moelcule is smaller.
This means that the nitrogen is diffusing into the ball much faster than the $SF_6$ is diffusing out of it.
For this reason we will have a situation in the meantime when a lot of nitrogen has already diffused into the ball but nearly all $SF_6$ is still in the ball.
In this situation the pressure in the ball is higher than in the initial situation (when only $SF_6$ is in the ball) and it is much higher than in the end of the process (when only air is left in the ball).
... to seemingly unbounded (until explosion) increase of pressure ...
If I didn't make a mistake, the increase of pressure cannot be more than the outside air pressure:
If the outside air pressure is about 1 bar and the initial pressure of the ball was 3 bars, the pressure of the ball will never exceed 4 bars.
This also means: A ball that does not explode when being exposed to a vacuum will also not explode because of this effect.
By the way:
The manufacturer should have filled the ball with a mixture of air and SF6: The amount of air that has the same volume as the ball at normal air pressure should have been mixed with the desired amount of $SF_6$.
In this case the number of air molecules diffusing into the ball is the same as the number of air molecules diffusing out of the ball.