It depends on what angle they're looking at, their state of motion, and the type of black hole. No, this is not controversial among people who have actually studied the problem carefully and understand relativity.
The effect you're talking about is identical to the Doppler shift. If you see light red-shifted from a certain direction in the sky, you are seeing signals from that direction slowed down. Blue shifts mean you're seeing signals sped up.
Here is a realistic simulation I did of an observer free-falling into a non-rotating black hole: https://www.youtube.com/watch?v=z-H-PipYCKc This observer is falling from rest, starting from far away.
What you'll see is that the stars observed in the outward(=backward) direction are red-shifted, while the ones observed at more forward angles are blueshifted. So stuff behind you is slowed down, while stuff ahead-ish and to the sides appears sped up. This is not actually super surprising, and is qualitatively similar to what you would see simply because of your motion, if there wasn't even a black hole there: blueshifts in the forward direction, redshifts in the backward direction.
Time dilation isn't really the best way to talk about these phenomena. When people talk about time dilation in a gravitational field, what they mean is something that's defined for a stationary observer. There can't be a stationary observer inside the horizon of a black hole.
Not to blow my own horn too much, but I've noticed that there are tons and tons of these "falling into a black hole" videos on youtube, and the popular ones that I sampled had tons of wrong physics, especially about this point that you're asking about.