In this paper I discuss the heuristic role which mathematics plays in physical discovery: first through the surplus structure which mathematics injects into physical principles which are given a mathematical formulation; secondly, through the realist interpretation of certain mathematical entities which appear at first sight to be devoid of any physical meaning. I then try to account for this dual role of mathematics in terms of a single philosophical principle, namely Meyerson's principle of identity. I finally apply these considerations to the study of two important questions; the questions namely of the continuity between STR and GTR (STR = Special Theory of Relativity, GTR = General Theory of Relativity) and of the emergence both of General Relativity and of the Unified Field Theories of Weyl, Eddington and Schrödinger-Einstein.