[HBASE-26838] Junit jar is not included in the hbase tar ball, causing issues for some hbase tools that do rely on it

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0, 3.0.0-alpha-2, 2.6.0, 2.4.11
• Fix Version/s: 2.5.0, 3.0.0-alpha-3, 2.4.12
• Component/s: integration tests, tooling
• Assignee: Wellington Chevreuil
• Reporter: Wellington Chevreuil
• Created: 14/Mar/22 22:02
• Updated: 01/Sep/22 10:57
• Resolved: 31/Mar/22 22:08

Description

We used to include junit jar on the generated tar ball lib directory. After some sanitisation of unnecessary libs for most of hbase processes, junit got removed from the packing, so that it don't end in hbase classpath by default. 

Some testing tools, however do depend on junit at runtime, and would now fail with NoClassDefFoundError, like IntegrationTestIngest:

2022-03-14T21:54:50,483 INFO  [main] client.AsyncConnectionImpl: Connection has been closed by main.
Exception in thread "main" java.lang.NoClassDefFoundError: org/junit/Assert
	at org.apache.hadoop.hbase.IntegrationTestIngest.initTable(IntegrationTestIngest.java:101)
	at org.apache.hadoop.hbase.IntegrationTestIngest.setUpCluster(IntegrationTestIngest.java:92)
	at org.apache.hadoop.hbase.IntegrationTestBase.setUp(IntegrationTestBase.java:170)
	at org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:153)
	at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:153)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.hbase.IntegrationTestIngest.main(IntegrationTestIngest.java:259)
Caused by: java.lang.ClassNotFoundException: org.junit.Assert
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	... 7 more 

Discussing with elserj internally, we believe a reasonable solution would be to include junit jar back into the tarball, under "lib/test" dir so that it's not automatically added to hbase processes classpath, but still allow operators to manually define it in a convenient way, like below:

HBASE_CLASSPATH="$HBASE_HOME/lib/tests/*" hbase org.apache.hadoop.hbase.IntegrationTesngest -m slowDeterministic 

Issue Links

• is related to: HBASE-26873 Nightly tests should exercise all of our public tools
• links to: GitHub Pull Request #4223