[HBASE-7934] TableMapReduceUtil doesn't include all dependency jars in new modular build

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.95.0, 0.95.2
• Fix Version/s: 0.98.0, 0.95.0
• Component/s: mapreduce
• Hadoop Flags: Reviewed
• Assignee: Nick Dimiduk
• Reporter: Nick Dimiduk
• Created: 25/Feb/13 22:57
• Updated: 23/Sep/13 18:31
• Resolved: 27/Feb/13 00:37

Description

From post to hbase-dev:

I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk. Presumably the old instructions for classpath assembly need an update. I guess I could create a hadoop jar that ships the classpath entries in lib, but IMHO that's not a viable solution for users who want to run one of the jobs we distribute. Am I missing something?

For you reference:

$ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2 -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv

...

Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
	at org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
	at org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
	at org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)

Issue Links

• is depended upon by: HBASE-7747 Import tools should use a combiner to merge Puts
• is related to: HBASE-7938 Add integration test for ImportTsv/LoadIncrementalHFiles workflow
• relates to: HBASE-7936 Undo prefix-tree module as dependency for mapreduce and for DataBlockEncoding