I will give a brief introduction to the physics of Pulsar Timing Arrays and then discuss their ability to test models of new physics. As an example, I will discuss how they can discover and/or constraint first-order cosmological phase transitions and models of Ultralight Dark Matter.