I ended up accomplishing this using a Custom HTML field & a little PHP and JS.
If anyone else ends up needing direction on this, here are a few links. This is not a drop-in solution for you but hopefully helps get you started.
PHP to handle an incoming AJAX request for term metadata & return JSON:
https://gist.github.com/21fed6266453de7ba76b3d2bc0e5a06f
JS to fetch the taxonomy term metadata and place it into the page:
https://gist.github.com/3b636d0c7e55ce6dc58ceba3de9a974e
Add Custom HTML field with html to support the incoming term metadata (I also give it a custom class):

After selecting a taxonomy it populates the Custom HTML like so:
