Here's my take at solving this. I temporarily substitute the context class loader to be the one from solr's resource loader (core).
This works (or seems to work) just fine. I don't particularly like it but I also don't want to mess with Solr's resource loaders if we don't have to.
I also piggyback a few cleanups.