XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When a Datanode comes online using Provided storage, it fills the ReplicaMap with the known replicas. With Provided Storage, this includes ProvidedReplica instances. Each of these objects, in their constructor, will construct an FileSystem using the Service Provider. This can result in contacting the remote file system and checking that the credentials are correct and that the data is there. For large systems this is a prohibitively expensive operation to perform per replica.

      Instead, the ProvidedVolumeImpl should own the reference to the remoteFS and should share it with the ProvidedReplica objects on their creation.

        Attachments

        1. HDFS-12093-HDFS-9806.001.patch
          9 kB
          Virajith Jalaparti
        2. HDFS-12093-HDFS-9806.002.patch
          9 kB
          Virajith Jalaparti

          Activity

            People

            • Assignee:
              virajith Virajith Jalaparti
              Reporter:
              ehiggs Ewan Higgs
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: