Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4058

DirectoryScanner may fail with IOOB if the directory scanning threads return out of volume order

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The DirectoryScanner may fail with an IOOB if the directory scanning threads return out of volume order (ie volume scanner #3 returns before volume scanner #2). This is because it's using an ArrayList and ArrayList#add(index, element) throws IOOB if idx >= size, and size is only increased as elements are added, therefore adds have to be done in index order. Since we know the size when we create the ArrayList let's just use an array (perhaps an ArrayList was used originally because someone thought array creation called the default constructor?).

        Attachments

        1. hdfs-4058.txt
          2 kB
          Eli Collins
        2. hdfs-4058.txt
          2 kB
          Eli Collins

          Activity

            People

            • Assignee:
              eli Eli Collins
              Reporter:
              eli Eli Collins

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment