Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9003

TableMapReduceUtil should not rely on org.apache.hadoop.util.JarFinder#getJar

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.2
    • Component/s: mapreduce
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This is the problem: TableMapReduceUtil#addDependencyJars relies on org.apache.hadoop.util.JarFinder if available to call getJar(). However getJar() uses File.createTempFile() to create a temporary file under hadoop.tmp.dir/target/test-dir. Due HADOOP-9737 the created jar and its content is not purged after the JVM is destroyed. Since most configurations point hadoop.tmp.dir under /tmp the generated jar files get purged by tmpwatch or a similar tool, but boxes that have hadoop.tmp.dir pointing to a different location not monitored by tmpwatch will pile up a collection of jars causing all kind of issues. Since JarFinder#getJar is not a public API from Hadoop (see Alejandro Abdelnur comment on HADOOP-9737) we shouldn't use that as part of TableMapReduceUtil in order to avoid this kind of issues.

        Attachments

        1. HBASE-9003.v3.patch
          7 kB
          Nick Dimiduk
        2. HBASE-9003.v2.patch
          26 kB
          stack
        3. HBASE-9003.v2.patch
          26 kB
          Esteban Gutierrez
        4. HBASE-9003.v1.patch
          25 kB
          Esteban Gutierrez
        5. HBASE-9003.v0.patch
          2 kB
          Esteban Gutierrez

          Activity

            People

            • Assignee:
              esteban Esteban Gutierrez
              Reporter:
              esteban Esteban Gutierrez
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: