I'm more with Dilaton than John Rennie here and I think the answer is kind of obvious, but you should take heed that a much greater thinker than I doesn't think so: see Wigner's famous article.
Now that you've been duly warned: in a nutshell mathematics can be thought of as the language one must naturally speak when trying to describe something objectively and without bias. We humans have many prejudices - our senses and particularly our brains have evolved in very specific conditions - those of the wet savannas of late neogene / early quternary Eastern Africa to recognize and react to and cope with the patterns that we encountered there. These are a very specific and very restricted set of patterns - not very representative of the World as a whole that physics seeks to describe and study. We need some way of rising above the prejudices that such a "sheltered" and "restricted" upbringing hinders us with.
One of the ways we do this is through abstraction in the sense of paring away all extraneous detail from problems we think about in physics. We "experimentally" find that there seems to be a repeatability and reliability in logical thought and description, in paring things back to sets of indivisible concepts (axioms) and then making deductions from them. These two actions of abstraction/ axiomatisation followed by deduction are roughly what mathematics is. Add to this a third action: checking our deductions /foretellings by experiment and you've got physics. Mathematics and physics are not greatly different in many ways. Mathematicians and physicists both seem to baulk and bridle at that thought, but I've never quite understood why. One is the language of the other. If you want to get into Goethe's head, then you'd be a fool not to study German thoroughly - likewise for physics and the need to study mathematics.
Interestingly, the idea of stripping extraneous detail away and seeing what the bare bones of a description leads to - an way of thinking that I'm sure many physicists can relate to - was given a special name and prosecuted most to its extreme to create whole new fields by a mathematician - this was the "Begriffliche Mathematik" (Conceptual Mathematics) of Emily Noether that begat great swathes of modern algebra, beginning with ideal theory in rings. The only difference between this kind of creation and that of physical theories is that we are obliged to check the latter with experiment - so that limits which ways deduction can go.
Many of the axiom systems of mathematics are grounded in very physical ideas, even though it may not be that obvious. Much abstract mathematics is created to "backfill" intuitive ideas about the physical world: a good example here is the theory of distribitions to lay a rigorous foundation for discussing ideas like the Dirac delta function - the idea of a fleetingly short, immensely intense pulse. You can trace even the most abstract and rarified mathematical ideas to ultimate physical world musings. The example I like is that of the mathematical notion of compactness and a compact set - one for which every open cover has a finite open subcover. Surely this one isn't a physical world idea? Actually, this notion was the last of many iterations of attempts to nail down what it was about the real numbers that gave rise to the physically / geometrically intuitive idea of uniform continuity. Various definitions can be seen - notably ones where compact is essentially taken to mean "having the Bolzano-Weierstass property" in the 1930s until the modern idea (proposed in 1935 by Alexandroff & Hopf) was generally settled on in the 1940s. It's a good example about initutive ideas being clarified by logical thought. The whole of real analysis ultimately comes from trying to pin down the intuitive idea of a continuous line with no "gaps" in it. These ideas come from our senses and our intuition for the World around us, but they are not always reliable. Mathematics helps us clarify these intuitions and sift out the reliable from the misleading - again, helps us overcome the prejudices of the wet savanna animal.
Footnote: I would post this question on Philosophy SE - there are some excellent thinkers there who are also mathematicians and physicists. In particular, I would be so bold as to summon up the user @NieldeBeaudrap to take a look - Neil is a researcher in quantum information who writes very thoughtful and interesting posts there.
Another footnote: readers should be warned that I am what many people would call pretty weird insofar that I pretty much subscribe to the Tegmark Mathematical Universe Hypothesis.