Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8870

NullPointerException when glob doesn't return files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.20.205.0, 1.0.3
    • Fix Version/s: 2.2.0
    • Component/s: fs, fs/s3
    • Labels:
      None

      Description

      Reading

      s3n://bucket/{a/,b/,c/}

      if one of the globs matches nothing, I get:

      Exception in thread "main" java.lang.NullPointerException
      	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:992)
      	at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:177)
      	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
      	at spark.HadoopRDD.<init>(HadoopRDD.scala:51)
      	at spark.SparkContext.hadoopFile(SparkContext.scala:186)
      	at spark.SparkContext.textFile(SparkContext.scala:155)
      	at com.celtra.analyzer.LogAnalyzer.analyzeSufficientS3Logs(LogAnalyzer.scala:52)
      	at com.celtra.analyzer.App$.main(App.scala:164)
      	at com.celtra.analyzer.App.main(App.scala)
      

      I'm not sure whether this is specific to S3 or all filesystems.

      This was occuring in 0.20.205 and I confirmed it's still present in 1.0.3.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jakajancar Jaka Jancar
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: