The big bang in relativity is not what you are imagining--- it isn't a localized explosion. You don't have stuff rushing out from a point, you have everything getting denser in the past in a homogenous way.
This is complicated a little by the fact that a Newtonian big bang has things rushing out from a single point. But even in a Newtonian bang everything is homogenous, all points look the same as all other points after a translation and a Galilean boost. The particles have a speed which is proportional to their distance, and this goes all the way to infinite speed in the Newtonian version.
But we don't live in a Newtonian universe, and when you have a relativistic big bang, the way it works is that all points are the same as all other points after a translation and a relativistic boost. At the point where the boost boosts you at the speed of light, so at the limit of the sphere you are imagining, you have a cosmological horizon. So it isn't that there is "perfect vacuum" beyond this sphere, there is a visibility boundary which marks the end of the universe as we can ever see it, and for a positivist (i.e. for a physicist) this makes it the boundary of the universe, period.
The sphere you are imagining is then not a boundary between stuff and vaccum, but it is the boundary of the entire universe, and there is nothing, not even vacuum, outside this sphere.
This is complicated a little by the fact that you can extend the solutions of General Relativity past horizons, so that you can imagine that there is extra space beyond the horizon, at least classically. In the extended model, the universe goes on beyond the horizon, and in a big-bang model, in a homogenous way, so that if you call "time" local time since the big bang, all places look the same at the same time. In an inflation model, you can have most of the external volume still inflating, so that the banging is at different "time" at different places, and in most places, counted by extended volume, the bang never happens. This point of view is called eternal inflation.
All these extended scenarios are just-so stories, since they either make no predictions for the observable universe (since by definition, we can't see the stuff outside the horizon), or they make statistical predictions based on saying that we live in a typical volume of the extended universe, and these statistical predictions are ridiculously wrong (they predict that inflation lasts as long as possible conditioned on us being here to observe what we observe, and this is false). These points of view attribute more information to the external universe than what can be encoded on the cosmological horizon, they are not compatible with causal patch complementarity and the holographic principle, and they should be considered dead.
The other answers to this question misinterpret your question. You are asking about a point explosion not having time to fill all of space, and this is just not how the big-bang works. The best way to talk about the relativistic big bang is to say that it happens everywhere, not at a point. The only caveat is that the Newtonian big-bang isn't like that, but ignore that for day-to-day intuition.
