Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14618

Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed
    • Patch

    Description

      I submitted a CR for this issue at:

      https://github.com/apache/hadoop/pull/1030

      The field timedOutItems (an ArrayList, i.e., not thread safe):

      https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70

      is protected by synchronization on itself (timedOutItems):

      https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168

      https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268

      https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178

      However, in one place:

      https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135

      it is (trying to be) protected by synchronized using pendingReconstructions — but this cannot protect timedOutItems.

      Synchronized on different objects does not ensure mutual exclusion with the other locations.

      I.e., 2 code locations, one synchronized by pendingReconstructions and the other by timedOutItems can still executed concurrently.

      This CR adds the synchronized on timedOutItems.

      Note that this CR keeps the synchronized on pendingReconstructions, which is needed for a different purpose (protect pendingReconstructions)

      Attachments

        1. race.patch
          0.9 kB
          Paul Ward

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            paulward24 Paul Ward
            paulward24 Paul Ward
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment