Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10820

Throw an exception in GenericOptionsParser when passed an empty Path

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None

      Description

      An empty token (e.g. "a.jar,,b.jar") in the -libjars option causes the current working directory to be recursively localized.

      Here's an example of this in action (using Hadoop 2.2.0):

      # create a temp directory and touch three JAR files
      mkdir -p tmp/path && cd tmp && touch a.jar b.jar c.jar path/d.jar
      
      # Run an example job only specifying two of the JARs.
      # Include an empty entry in libjars.
      hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi -libjars a.jar,,c.jar 2 1000000000
      
      # As the job is running examine the localized directory in HDFS.
      # Notice that not only are the two JAR's specified in libjars copied,
      # but in addition the contents of the working directory are also recursively copied.
      $ hadoop fs -lsr /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/a.jar
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/c.jar
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/a.jar
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/b.jar
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/c.jar
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/path
      /tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/path/d.jar
      

        Attachments

        1. HADOOP-10820-3.patch
          5 kB
          Zhihai Xu
        2. HADOOP-10820-2.patch
          5 kB
          Zhihai Xu
        3. HADOOP-10820-1.patch
          7 kB
          Alex Holmes
        4. HADOOP-10820.patch
          3 kB
          Alex Holmes

          Activity

            People

            • Assignee:
              zxu Zhihai Xu
              Reporter:
              alex.holmes Alex Holmes
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: