I ended up accomplishing this using a Custom HTML field & a little PHP and JS.
If anyone else ends up needing direction on this, here are a few links. This is not a drop-in solution for you but hopefully helps get you started.
PHP to handle an incoming AJAX request for term metadata & return JSON:
JS to fetch the taxonomy term metadata and place it into the page:
Add Custom HTML field with html to support the incoming term metadata (I also give it a custom class):
After selecting a taxonomy it populates the Custom HTML like so: