The problem is this statement: "In order to see the event horizon’s radius increase, you first have to see mass fall into it." The truth is that you can never see a black hole's event horizon from the outside, because it is always in the future. A black hole twists time and space round so that on the event horizon the whole of an observer's future time (his future light cone) points into the hole. You can no more see the event horizon than you can see tomorrow. What you can see from the outside is the effect of the black hole's gravity on light rays passing through the space surrounding the event horizon, in the past.
When you look directly towards a black hole, the rays of light you see come from the distant past, from all the way back to when the black hole was formed. It only looks black because a few seconds of outgoing light from the last few seconds of an object's fall gets stretched out over millions of years. It has been struggling to climb out of the gravity well for all this time. If you spread the energy from a few seconds of light rays out over millions of years, it looks dark.
When mass falls into the hole, the hole gets bigger, the gravity outside the hole gets stronger, and there is more distortion of passing light rays further out from the hole. The region where light rays get distorted appears bigger. And the 'black circle' you see in the middle (containing spread out light from the distant past) gets bigger. But you are not seeing the horizon itself - only light that came uncomfortably close and managed to escape.