[PHOENIX-6312] Need a util method in PhoenixMapReduceUtil along the lines of TableMapReduceUtil.addHBaseDependencyJars

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 5.1.0
• Assignee: Unassigned
• Reporter: Jacob Isaac
• Created: 13/Jan/21 02:55
• Updated: 25/Feb/21 02:43
• Resolved: 01/Feb/21 09:54

Description

Now that we have phoenix-hbase-compat-x-x-x jars, We need to have the classes in the compat jar made available to the MR jobs.

TableMapReduceUtil.addHBaseDependencyJars is an example of how hbase dependency jars are made available to the MR job.

We get the following exception when these jars are not made available to MR jobs
Error: java.lang.ClassNotFoundException: org.apache.phoenix.compat.hbase.CompatRpcControllerFactory at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.phoenix.query.QueryServicesOptions.<clinit>(QueryServicesOptions.java:288) at org.apache.phoenix.query.QueryServicesImpl.<init>(QueryServicesImpl.java:36) at org.apache.phoenix.jdbc.PhoenixDriver.getQueryServices(PhoenixDriver.java:197) at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:235) at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:142) at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:113) at org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:58) at org.apache.phoenix.mapreduce.PhoenixServerBuildIndexInputFormat.getQueryPlan(PhoenixServerBuildIndexInputFormat.java:94) at org.apache.phoenix.mapreduce.PhoenixInputFormat.createRecordReader(PhoenixInputFormat.java:79) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:521) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
 

Issue Links

• is fixed by: PHOENIX-6307 Build and release official binary distributions with each HBase profile