A very good resource that treats a great number of physical topics assuming some degree of mathematical maturity is Gustafson and Sigal's Mathematical Concepts of Quantum Mechanics. Despite the title, the book isn't your typical rigorous resource aimed at mathematicians. I quote the following from the book's preface:
Given our time constraint, we have often pursued mathematical content
at the expense of rigor. However, wherever we have sacrificed the latter, we have tried to explain whether the result is an established fact, or, mathematically speaking, a conjecture, and in the former case, how a given argument can be made rigorous.
This book clearly meets your needs as it requires some pre-requisite knowledge in mathematics (albeit very briefly reviewed if necessary) as mentioned in the preface, which i quote:
Prerequisites for this book are introductory real analysis (notions of vector space, scalar product, norm, convergence, Fourier transform) and complex analysis, the theory of Lebesgue integration, and elementary differential
equations. These topics are typically covered by the third year in mathematics
departments. The first and third topics are also familiar to physics undergraduates. However, even in dealing with mathematics students we have found it useful, if not necessary, to review these notions, as needed for the course. Hence, to make the book relatively self-contained, we briefly cover these subjects, with the exception of Lebesgue integration. Those unfamiliar with the latter can think about Lebesgue integrals as if they were Riemann integrals. This said, the pace of the book is not a leisurely one and requires, at least for beginners, some amount of work.
This book covers a vast variety of topics that are not found in your typical quantum mechanics textbook along with topics that usually are present as well. It would be prudent to use this book as a primary resource despite the slightly more difficult pace due to the clarity of exposition.
