Physics is like carpentry: Using the right tool makes the job easier, not more difficult. David J. Griffiths in his book "Quantum Mechanics"
Although this is a travel guide to physics, you'll find here, of course, lots of pages about mathematics.
Mathematics is the "language of nature" and physics isn't possible without it. However, in contrast to the usual Wikipedia pages about math topics, our approach can be summarized as follows: "Mathematics is a toolbox and we pick only those tools that are useful in physics".
Especially, this means that for every mathematical concept explained here, there is a "Why is it useful?" section that explains where and how the concept is useful in physics.
Mathematics is a part of physics. Physics is an experimental science, a part of natural science. Mathematics is the part of physics where experiments are cheap.