Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5697

Audit HBase for usage of deprecated hadoop 0.20.x property names.

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      All Hadoop 0.20.x and 1.x specific configuration settings have been removed and replaced with the new Hadoop 2.x configuration property names. If you use any of these in your command line scripts or xml configuration files, you will need to use the new properties. These include:

      hadoop.native.lib -> io.native.lib.available
      mapred.job.classpath.archives -> mapreduce.job.classpath.archives
      mapred.map.tasks.speculative.execution -> mapreduce.map.speculative
      mapred.task.id -> mapreduce.task.attempt.id
      mapred.output.compress -> mapreduce.output.fileoutputformat.compress
      mapred.output.compression.codec -> mapreduce.output.fileoutputformat.compress.codec
      mapred.output.compression.type -> mapreduce.output.fileoutputformat.compress.type
      mapred.reduce.tasks.speculative.execution -> mapreduce.reduce.speculative
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      mapred.job.name -> mapreduce.job.name
      mapred.local.dir -> mapreduce.cluster.local.dir
      mapred.temp.dir -> mapreduce.cluster.temp.dir
      mapred.system.dir -> mapreduce.jobtracker.system.dir
      mapred.working.dir -> mapreduce.job.working.dir
      mapred.job.tracker -> mapreduce.jobtracker.address
      heartbeat.recheck.interval -> dfs.namenode.heartbeat.recheck-interval
      dfs.socket.timeout -> dfs.client.socket-timeout
      dfs.block.size -> dfs.blocksize
      io.sort.mb -> mapreduce.task.io.sort.mb
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      min.num.spills.for.combine -> mapreduce.map.combine.minspills
      mapred.map.max.attempts -> mapreduce.map.max.attempts
      dfs.socket.timeout -> dfs.client.socket-timeout
      dfs.datanode.max.xcievers -> dfs.datanode.max.transfer.threads

      Show
      All Hadoop 0.20.x and 1.x specific configuration settings have been removed and replaced with the new Hadoop 2.x configuration property names. If you use any of these in your command line scripts or xml configuration files, you will need to use the new properties. These include: hadoop.native.lib -> io.native.lib.available mapred.job.classpath.archives -> mapreduce.job.classpath.archives mapred.map.tasks.speculative.execution -> mapreduce.map.speculative mapred.task.id -> mapreduce.task.attempt.id mapred.output.compress -> mapreduce.output.fileoutputformat.compress mapred.output.compression.codec -> mapreduce.output.fileoutputformat.compress.codec mapred.output.compression.type -> mapreduce.output.fileoutputformat.compress.type mapred.reduce.tasks.speculative.execution -> mapreduce.reduce.speculative mapred.input.dir -> mapreduce.input.fileinputformat.inputdir mapred.job.name -> mapreduce.job.name mapred.local.dir -> mapreduce.cluster.local.dir mapred.temp.dir -> mapreduce.cluster.temp.dir mapred.system.dir -> mapreduce.jobtracker.system.dir mapred.working.dir -> mapreduce.job.working.dir mapred.job.tracker -> mapreduce.jobtracker.address heartbeat.recheck.interval -> dfs.namenode.heartbeat.recheck-interval dfs.socket.timeout -> dfs.client.socket-timeout dfs.block.size -> dfs.blocksize io.sort.mb -> mapreduce.task.io.sort.mb mapred.input.dir -> mapreduce.input.fileinputformat.inputdir mapred.input.dir -> mapreduce.input.fileinputformat.inputdir min.num.spills.for.combine -> mapreduce.map.combine.minspills mapred.map.max.attempts -> mapreduce.map.max.attempts dfs.socket.timeout -> dfs.client.socket-timeout dfs.datanode.max.xcievers -> dfs.datanode.max.transfer.threads

      Description

      Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes.

      Here is a list of the hadoop property name changes:

      hadoop.native.lib -> io.native.lib.available
      mapred.job.classpath.archives -> mapreduce.job.classpath.archives
      mapred.map.tasks.speculative.execution -> mapreduce.map.speculative
      mapred.task.id -> mapreduce.task.attempt.id
      mapred.output.compress -> mapreduce.output.fileoutputformat.compress
      mapred.output.compression.codec -> mapreduce.output.fileoutputformat.compress.codec
      mapred.output.compression.type -> mapreduce.output.fileoutputformat.compress.type
      mapred.reduce.tasks.speculative.execution -> mapreduce.reduce.speculative
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      mapred.job.name -> mapreduce.job.name
      mapred.local.dir -> mapreduce.cluster.local.dir
      mapred.temp.dir -> mapreduce.cluster.temp.dir
      mapred.system.dir -> mapreduce.jobtracker.system.dir
      mapred.working.dir -> mapreduce.job.working.dir
      mapred.job.tracker -> mapreduce.jobtracker.address
      heartbeat.recheck.interval -> dfs.namenode.heartbeat.recheck-interval
      dfs.socket.timeout -> dfs.client.socket-timeout
      dfs.block.size -> dfs.blocksize
      io.sort.mb -> mapreduce.task.io.sort.mb
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      mapred.input.dir -> mapreduce.input.fileinputformat.inputdir
      min.num.spills.for.combine -> mapreduce.map.combine.minspills
      mapred.map.max.attempts -> mapreduce.map.max.attempts
      dfs.socket.timeout -> dfs.client.socket-timeout
      dfs.datanode.max.xcievers -> dfs.datanode.max.transfer.threads

        Attachments

        1. deprecated_properties
          1 kB
          Srikanth Srungarapu
        2. HBASE-5697_v2.patch
          49 kB
          Srikanth Srungarapu
        3. HBASE-5697_v3.patch
          49 kB
          Srikanth Srungarapu
        4. HBASE-5697.patch
          49 kB
          Srikanth Srungarapu
        5. hbase-5697.v3.patch
          46 kB
          Jonathan Hsieh

          Issue Links

            Activity

              People

              • Assignee:
                srikanth235 Srikanth Srungarapu
                Reporter:
                jmhsieh Jonathan Hsieh
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: