Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4999

IndexOutOfBoundsException in FSEditLog

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.20.0
    • None
    • None
    • Reviewed

    Description

      when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
      int numEditStreams = editStreams.size();
      for (int idx = 0; idx < numEditStreams(); idx++) {
      ...
      processIOError(idx);
      ...
      }

      but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.

      proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).

      Attachments

        1. HADOOP-4999.patch
          1 kB
          Boris Shkolnik
        2. HADOOP-4999-1.patch
          1 kB
          Boris Shkolnik

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            boryas Boris Shkolnik
            boryas Boris Shkolnik
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment