Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2541

For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.1, 1.1.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Running off 0.20-security, I noticed that one could get the following exception when scanners are used:

      DataXceiver 
      java.lang.IllegalArgumentException: n must be positive 
      at java.util.Random.nextInt(Random.java:250) 
      at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.getNewBlockScanTime(DataBlockScanner.java:251) 
      at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.addBlock(DataBlockScanner.java:268) 
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:432) 
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:122)
      

      This is cause the period, determined in the DataBlockScanner (0.20+) or BlockPoolSliceScanner (0.23+), is cast to an integer before its sent to a Random.nextInt(...) call. For sufficiently large values of the long 'period', the casted integer may be negative. This is not accounted for. I'll attach a sample test that shows this possibility with the numbers.

      We should ensure we do a Math.abs(...) before we send it to the Random.nextInt(...) call to avoid this.

      With this bug, the maximum # of blocks a scanner may hold in its blocksMap without opening up the chance for beginning this exception (intermittent, as blocks continue to grow) would be 3582718.

      1. HDFS-2541.patch
        1 kB
        Harsh J
      2. BSBugTest.java
        0.5 kB
        Harsh J

        Issue Links

          Activity

            People

            • Assignee:
              Harsh J
              Reporter:
              Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development