Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3064

Exception with file globbing closures

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.16.1
    • 0.17.0
    • None
    • None

    Description

      Using file globbing to select various input paths, like so:

      conf.setInputPath(new Path("mr/input/glob/2008/02/

      {02,08}

      "));

      gives an exception:

      Exception in thread "main" java.io.IOException: Illegal file pattern:
      Expecting set closure character or end of range, or } for glob {02 at
      3
      at org.apache.hadoop.fs.FileSystem$GlobFilter.error(FileSystem.java:1023)
      at org.apache.hadoop.fs.FileSystem$GlobFilter.setRegex(FileSystem.java:1008)
      at org.apache.hadoop.fs.FileSystem$GlobFilter.<init>(FileSystem.java:926)
      at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:826)
      at org.apache.hadoop.fs.FileSystem.globPaths(FileSystem.java:873)
      at org.apache.hadoop.mapred.FileInputFormat.validateInput(FileInputFormat.java:131)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:541)
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:809)

      The code for JobConf.getInputPaths tokenizes using
      a comma as the delimiter, producing two paths
      "mr/input/glob/2008/02/

      {02" and "08}

      ".

      Attachments

        1. inputPath.patch
          7 kB
          Hairong Kuang
        2. inputPath1.patch
          8 kB
          Hairong Kuang
        3. inputPath2.patch
          13 kB
          Hairong Kuang
        4. inputPath3.patch
          13 kB
          Hairong Kuang
        5. inputPath4.patch
          13 kB
          Thomas White

        Activity

          People

            hairong Hairong Kuang
            tomwhite Thomas White
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: