Roman Shaposhnik I think that would be great to do eventually, but it seems like for the immediate goal of simplifying JDBC driver installation, it's a bit much. I definitely think we should implement this solution with an eye toward making it more comprehensive in the future, though.
Attached is an alternate idea that does two things:
1. Search for these libraries in places where distros commonly put them (just /usr/share/java for now), and creates symlinks to them in /var/lib/bigtop. This behavior is easily disabled in /etc/defaults/bigtop-utils, and will not overwrite existing symlinks.
2. Adds the contents of /var/lib/bigtop to the classpath for Hive and Oozie (the latter I was unable to fully verify because of some other problems, but it appears correct).
I'd like to get some thoughts on this approach, and will then apply it to other components as appropriate (I think Sqoop 2, when committed, will be the only other one that would use this immediately).
Some other ideas for improvement include component-specific sub-directories (although we already have that with /usr/lib/*, IMHO), and perhaps adding libraries to the classpath in reverse-alphabetical order to ensure higher versions take precedence, and something plugins.d-like later.
Thoughts on this approach? I've heard concerns about things ending up in the classpath that shouldn't, so maybe we want to disable the symlinking by default (although I like the fact that in distros that ship it, the mysql-connector-java package will work for Bigtop out-of-the-box), and we should definitely document it's intended purpose well.