Powerful jets hosted by accreting super-massive black holes have long been candidates for the acceleration sites for high-energy extra-galactic cosmic rays, supported by the recent association of neutrinos from blazar TX0506+056. In the highly-aligned jets known as blazars, the X-ray to TeV radiation is usually attributed to inverse Compton scattering processes, but has not been clearly identified in most cases due to degeneracies in physical models. AP Librae, a blazar detected in TeV energies, has an extremely broad high-energy spectrum, covering ∼ 9 decades in energy. Using new ALMA and Hubble imaging of the kpc-scale jet and over 11 years of Fermi/LAT observations, we rule out previously proposed leptonic models attributing the high-energy emission to synchrotron self-Compton from the jet base and IC/CMB in the kpc-scale jet. In contrast, "lepto-hadronic" models remain viable, though underconstrained given the number of free parameters. We find that the origin of the TeV photons from this source remains debatable and show that leptonic and hadronic models can be further tested with deep and high dynamic range imaging in the sub-mm and far infrared and/or continued monitoring of the source at TeV energies to test for variability. Unmasking the origin of extragalactic TeV emission from blazar AP Librae would unlock vital clues to our understanding of particle acceleration and the origin of extra-galactic cosmic rays.