[HBASE-5697] Audit HBase for usage of deprecated hadoop 0.20.x property names.

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.99.0
• Hadoop Flags: Incompatible change, Reviewed

Release Note

All Hadoop 0.20.x and 1.x specific configuration settings have been removed and replaced with the new Hadoop 2.x configuration property names. If you use any of these in your command line scripts or xml configuration files, you will need to use the new properties. These include:

hadoop.native.lib -> io.native.lib.available
mapred.job.classpath.archives -> mapreduce.job.classpath.archives
mapred.map.tasks.speculative.execution -> mapreduce.map.speculative
mapred.task.id -> mapreduce.task.attempt.id
mapred.output.compress -> mapreduce.output.fileoutputformat.compress
mapred.output.compression.codec -> mapreduce.output.fileoutputformat.compress.codec
mapred.output.compression.type -> mapreduce.output.fileoutputformat.compress.type
mapred.reduce.tasks.speculative.execution -> mapreduce.reduce.speculative
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
mapred.job.name -> mapreduce.job.name
mapred.local.dir -> mapreduce.cluster.local.dir
mapred.temp.dir -> mapreduce.cluster.temp.dir
mapred.system.dir -> mapreduce.jobtracker.system.dir
mapred.working.dir -> mapreduce.job.working.dir
mapred.job.tracker -> mapreduce.jobtracker.address
heartbeat.recheck.interval -> dfs.namenode.heartbeat.recheck-interval
dfs.socket.timeout -> dfs.client.socket-timeout
dfs.block.size -> dfs.blocksize
io.sort.mb -> mapreduce.task.io.sort.mb
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
min.num.spills.for.combine -> mapreduce.map.combine.minspills
mapred.map.max.attempts -> mapreduce.map.max.attempts
dfs.socket.timeout -> dfs.client.socket-timeout
dfs.datanode.max.xcievers -> dfs.datanode.max.transfer.threads

• Assignee: Srikanth Srungarapu
• Reporter: Jonathan Hsieh
• Created: 02/Apr/12 04:38
• Updated: 21/Feb/15 23:35
• Resolved: 29/Apr/14 18:19

Description

Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes.

Here is a list of the hadoop property name changes:

hadoop.native.lib -> io.native.lib.available
mapred.job.classpath.archives -> mapreduce.job.classpath.archives
mapred.map.tasks.speculative.execution -> mapreduce.map.speculative
mapred.task.id -> mapreduce.task.attempt.id
mapred.output.compress -> mapreduce.output.fileoutputformat.compress
mapred.output.compression.codec -> mapreduce.output.fileoutputformat.compress.codec
mapred.output.compression.type -> mapreduce.output.fileoutputformat.compress.type
mapred.reduce.tasks.speculative.execution -> mapreduce.reduce.speculative
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
mapred.job.name -> mapreduce.job.name
mapred.local.dir -> mapreduce.cluster.local.dir
mapred.temp.dir -> mapreduce.cluster.temp.dir
mapred.system.dir -> mapreduce.jobtracker.system.dir
mapred.working.dir -> mapreduce.job.working.dir
mapred.job.tracker -> mapreduce.jobtracker.address
heartbeat.recheck.interval -> dfs.namenode.heartbeat.recheck-interval
dfs.socket.timeout -> dfs.client.socket-timeout
dfs.block.size -> dfs.blocksize
io.sort.mb -> mapreduce.task.io.sort.mb
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
min.num.spills.for.combine -> mapreduce.map.combine.minspills
mapred.map.max.attempts -> mapreduce.map.max.attempts
dfs.socket.timeout -> dfs.client.socket-timeout
dfs.datanode.max.xcievers -> dfs.datanode.max.transfer.threads

Issue Links

• is duplicated by: HBASE-5554 "hadoop.native.lib" config is deprecated
• is related to: HBASE-5738 Using HBase with HA HDFS requires bogus hardcoded port value
• relates to: HBASE-4924 More issues running hbase on hadoop 0.23