What's the actual reason for him accelerating
The reason is that when the bus slows down, he is not slowed down along with it.
The bus and him have the same speed to start with. When the bus slows down, it looks like it moves backwards from his perspective. He still continues forwards with the same speed, because nothing stops him.
The human brain might misinterpret this, though. It might be more intuitive to believe that the bus is stationary and that you are the one moving, rather than you being stationary and the bus moving backwards. But this is just a wrong interpretation of what the eyes see. It is the bus slowing down, and not himself speeding up.
Regarding your further questions in a comment, I changed my reply to an answer here:
[...] what's the cause of jerk we feel when driver applies the break
You don't feel any jerk. Everyone else do (backwards).
A person sitting in a seat with a seat belt on wants to continue moving at constant speed, just like the man on roller skates. But the seat belt catches him and pulls him backwards along with the bus. It is not him moving into the seat belt but rather the seat belt slowing down and moving backwards into him.
[...] why i moved forward when driver applies the break
You continue forwards since nothing slows you down. As you are explaining already, no force acts in you. Only the bus slows down. It looks like you are pushed forwards while it actually is the bus that is being pushed backwards.
I want to understand this from the frame of reference of bus
The explanation is the same regardless of the frame you see it from. The World works the same no matter where you are in it.
But Newton's laws do not hold true in non-inertial frames. So if you choose such a frame it all gets tricky. Then you must invent some "fictitious" forces or "pseudo" forces that do not exist in reality before Newton's laws make sense again.
These fictitious forces just represent this tendency to continue moving, that we have explained above.
No matter what frame you choose, the explanation about what in reality is taking place in this situation is the same.
From earth inertia is the explanation but what from bus
Actually, inertia is not the explanation. Inertia (mass) is a resistance to accelerate. But the reason that you fly forwards in a braking bus is that you are not accelerated (negatively) at all!
It is not that you are resisting deceleration, but rather that you are not being decelerated.
If you are decelerated, such as the passenger with a seat belt on, then it is inertia that determines how tough it is to slow you down (how much tension the belt must provide and how squeezed you feel).