I have come across lectures about Higher Topos Theory in mathematical physics and I am wondering about the explicit restriction to the category of smooth sets.
Why should the potentially possible laws of physics be restricted to live inside category of smooth sets? Is this category chosen because of experiments that indicate that we live in a "smooth" world? Could alternate possible laws of physics (by possible I mean internally consistent) that describe some other possible universe not constrained by relativity and QM "live inside" much less restricted set theoretical universe $V$?
Edit: By smooth set I mean a generalized smooth space that may be probed by smooth Cartesian spaces (https://ncatlab.org/nlab/show/smooth+set).
Lecture by Urs Schreiber: https://www.youtube.com/watch?v=Gr_BCr1rYVM&t=1791s