Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      java.lang.IllegalArgumentException: n must be positive
      at java.util.Random.nextInt(Random.java:250)
      at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243)
      at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:289)
      at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
      at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1221)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1129)
      at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:549)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:623)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
      at org.apache.hadoop.mapred.Child.main(Child.java:159)

      confChanged assumes that the list of dirs it creates (LocalDirAllocator.java:215) has at least one element in it by the end of the function. If, for each local dir, either the conditional on line 221 is false, or the call to DiskChecker.checkDir() throws an exception, this assumption will not hold. In this case, dirIndexRandomizer.nextInt() is called on the number of elements in dirs, which is 0. Since dirIndexRandomizer (195) is an instance of Random(), it needs a positive (non-zero) argument to nextInt().

        Issue Links

          Activity

          Peter Arthur Ciccolo created issue -
          Hide
          Koji Noguchi added a comment -

          Peter, if you're using yahoo clusters it could be ops accidentally deleting the local dirs after TaskTracker came up.

          Show
          Koji Noguchi added a comment - Peter, if you're using yahoo clusters it could be ops accidentally deleting the local dirs after TaskTracker came up.
          Dave Latham made changes -
          Field Original Value New Value
          Link This issue duplicates MAPREDUCE-635 [ MAPREDUCE-635 ]
          Allen Wittenauer made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Cannot Reproduce [ 5 ]
          Hide
          chongyuanhuang added a comment -

          May be, It's taken by something about "local dirs". But how to solve this problem?

          Show
          chongyuanhuang added a comment - May be, It's taken by something about "local dirs". But how to solve this problem?
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          1529d 20h 47m 1 Allen Wittenauer 22/Jul/14 20:30

            People

            • Assignee:
              Unassigned
              Reporter:
              Peter Arthur Ciccolo
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development