The noise coming out of the heating water can be caused by the formation of air bubbles which are dissolved gases that come out as the water heats. Finally as the water nears boiling, bubbles of steam formed in the bulk of the water also produce sound but they cool as they rise and burst leaving cavities which the water fills up also releasing sound. This process is known as cavitation and also produces sound. Once all the water begins to boil, the sound subsides as cavitation decreases.
When the kettle is turned off the heat supply stops and as water continues to boil and steam escapes for a very short period, the liquid water is cooled and the temperature soon falls below the boiling point and the formation of bubbles stops.