Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-13110

[SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: HDFS-10285, 3.2.0
    • Component/s: None
    • Labels:
      None

      Description

      This task is to address the following Daryn Sharp's comments. Please refer HDFS-10285 to see more detailed discussion.

      Comment-10)

      NamenodeProtocolTranslatorPB

      Most of the api changes appear unnecessary.

      IntraSPSNameNodeContext#getFileInfo swallows all IOEs, based on assumption that any and all IOEs means FNF which probably isn’t the intention during rpc exceptions.

       Comment-13)

      StoragePolicySatisfier
      It appears to make back-to-back calls to hasLowRedundancyBlocks and getFileInfo for every file. Haven’t fully groked the code, but if low redundancy is not the common case, then it shouldn’t be called unless/until needed. It looks like files that are under replicated are re-queued again?

        Attachments

        1. HDFS-13110-HDFS-10285-05.patch
          117 kB
          Rakesh Radhakrishnan
        2. HDFS-13110-HDFS-10285-04.patch
          100 kB
          Rakesh Radhakrishnan
        3. HDFS-13110-HDFS-10285-03.patch
          91 kB
          Rakesh Radhakrishnan
        4. HDFS-13110-HDFS-10285-02.patch
          86 kB
          Rakesh Radhakrishnan
        5. HDFS-13110-HDFS-10285-01.patch
          86 kB
          Rakesh Radhakrishnan
        6. HDFS-13110-HDFS-10285-00.patch
          86 kB
          Rakesh Radhakrishnan

          Activity

            People

            • Assignee:
              rakeshr Rakesh Radhakrishnan
              Reporter:
              rakeshr Rakesh Radhakrishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: