[HBASE-15129] Set default value for hbase.fs.tmp.dir rather than fully depend on hbase-default.xml

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0, 1.3.0, 1.1.4, 1.0.4, 0.98.18, 2.0.0
• Component/s: mapreduce
• Hadoop Flags: Reviewed

Release Note

Before HBASE-15129, if somehow hbase-default.xml is not on classpath, default values for hbase.fs.tmp.dir and hbase.bulkload.staging.dir are left empty. After HBASE-15129, default values of both properties are set to "/user/<user.name>/hbase-staging".

• Assignee: Yu Li
• Reporter: Yu Li
• Created: 19/Jan/16 12:13
• Updated: 15/Feb/16 04:01
• Resolved: 03/Feb/16 01:15

Description

One of our users has observed below error when our cluster upgrades from 0.98.12 to 1.1.2:

java.lang.IllegalArgumentException: Can not create a Path from a null string
        at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
        at org.apache.hadoop.fs.Path.<init>(Path.java:134)
        at org.apache.hadoop.fs.Path.<init>(Path.java:88)
        at org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:639)
        at org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:489)

And checking the application code:

  private Job createJob(Configuration jobConf) throws IOException {
    String tableName = jobConf.get(TABLE_NAME_KEY);
    Job job = new Job(jobConf);
    Configuration tableConf = HBaseConfiguration.create();
    HTable table = new HTable(tableConf, tableName);
    HFileOutputFormat2.configureIncrementalLoad(job, table);
    return job;
  }

We could see the user has his own hbase-independent job configuration, so our current code in HFileOutputFormat2:

Path partitionsPath = new Path(conf.get("hbase.fs.tmp.dir"), "partitions_" + UUID.randomUUID());

will return a null string and cause the above exception.

We propose to add default value in the code rather than depend on hbase-default.xml in this JIRA.

This is an improvement for HBASE-13625