Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2541

For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.1, 1.1.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Running off 0.20-security, I noticed that one could get the following exception when scanners are used:

      DataXceiver 
      java.lang.IllegalArgumentException: n must be positive 
      at java.util.Random.nextInt(Random.java:250) 
      at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.getNewBlockScanTime(DataBlockScanner.java:251) 
      at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.addBlock(DataBlockScanner.java:268) 
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:432) 
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:122)
      

      This is cause the period, determined in the DataBlockScanner (0.20+) or BlockPoolSliceScanner (0.23+), is cast to an integer before its sent to a Random.nextInt(...) call. For sufficiently large values of the long 'period', the casted integer may be negative. This is not accounted for. I'll attach a sample test that shows this possibility with the numbers.

      We should ensure we do a Math.abs(...) before we send it to the Random.nextInt(...) call to avoid this.

      With this bug, the maximum # of blocks a scanner may hold in its blocksMap without opening up the chance for beginning this exception (intermittent, as blocks continue to grow) would be 3582718.

        Attachments

        1. HDFS-2541.patch
          1 kB
          Harsh J
        2. BSBugTest.java
          0.5 kB
          Harsh J

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                qwertymaniac Harsh J
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: