[CRUNCH-245] hbase.zookeeper.quorum is overriden by hbase-default.xml

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Component/s: IO
• Assignee: Chao Shi
• Reporter: Chao Shi
• Created: 02/Aug/13 06:20
• Updated: 08/Nov/13 21:24
• Resolved: 02/Aug/13 14:35

Description

I'm using crunch-hbase on a small cluster with 4 nodes. The job complains cannot talk to ZK at localhost. In fact, I specify hbase.zookeeper.quorum programmatically, but it seems it is not passed to the job (as observed in JT's webpage).

After some investigation, I found the cause is the following code. I have no idea why addHBaseResources is called here. Anybody knows? After comment it out, the job can read data from HBase.

@Override
public void configureSource(Job job, int inputId) throws IOException {
Configuration conf = job.getConfiguration();
HBaseConfiguration.addHbaseResources(conf);
TableMapReduceUtil.addDependencyJars(job);
if (inputId == -1)

{ job.setMapperClass(CrunchMapper.class); job.setInputFormatClass(inputBundle.getFormatClass()); inputBundle.configure(job.getConfiguration()); }

else

{ Path dummy = new Path("/hbase/" + table); CrunchInputs.addInputPath(job, dummy, inputBundle, inputId); }

}