Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14986

ReplicaCachingGetSpaceUsed throws ConcurrentModificationException

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Running DU across lots of disks is very expensive . We applied the patch HDFS-14313 to get  used space from ReplicaInfo in memory.However, new du threads throw the exception

      // 2019-11-08 18:07:13,858 ERROR [refreshUsed-/home/vipshop/hard_disk/7/dfs/dn/current/BP-1203969992-XXXX-1450855658517] org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed: ReplicaCachingGetSpaceUsed refresh error
      java.util.ConcurrentModificationException: Tree has been modified outside of iterator    
      at org.apache.hadoop.hdfs.util.FoldedTreeSet$TreeSetIterator.checkForModification(FoldedTreeSet.java:311)    
      at org.apache.hadoop.hdfs.util.FoldedTreeSet$TreeSetIterator.hasNext(FoldedTreeSet.java:256)    
      at java.util.AbstractCollection.addAll(AbstractCollection.java:343)    
      at java.util.HashSet.<init>(HashSet.java:120)    
      at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.deepCopyReplica(FsDatasetImpl.java:1052)    
      at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed.refresh(ReplicaCachingGetSpaceUsed.java:73)    
      at org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:178)   
      at java.lang.Thread.run(Thread.java:748)
      

        Attachments

        1. HDFS-14986.006.patch
          9 kB
          Aiphago
        2. HDFS-14986.005.patch
          9 kB
          Aiphago
        3. HDFS-14986.004.patch
          9 kB
          Aiphago
        4. HDFS-14986.003.patch
          8 kB
          Aiphago
        5. HDFS-14986.002.patch
          8 kB
          Aiphago
        6. HDFS-14986.001.patch
          3 kB
          Aiphago

        Issue Links

          Activity

            People

            • Assignee:
              Aiphag0 Aiphago
              Reporter:
              jianliang.wu Ryan Wu

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment