I make digital copies of CD/DVD/Vinyls I buy and use those copies myself (on my phone/tablet/laptop). I am free to use those copies because I purchased the physical version.
If I lose or break the physical version am I still allowed to use the digital versions?
Links in French, sorry, I don't know Dutch/Flemish.
Yes, the law allowing you to make the copies in the first place doesn't require maintaining the source.
Article XI.190 of the Code de droit économique:
Lorsque l'oeuvre a été licitement divulguée, l'auteur ne peut interdire : [...] 9° la reproduction d'oeuvres, à l'exception des partitions musicales, effectuée dans le cercle de famille et réservée à celui-ci;
Roughly translated,
Once the work has been legally distributed, the author may not forbid: [...] 9. The reproduction of works, with the exception of sheet music, made within and reserved for the family circle.
For more on private copy in Belgium, see this government resource.
