Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-13110

[SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-10285, 3.2.0
    • None
    • None

    Description

      This task is to address the following daryn's comments. Please refer HDFS-10285 to see more detailed discussion.

      Comment-10)

      NamenodeProtocolTranslatorPB

      Most of the api changes appear unnecessary.

      IntraSPSNameNodeContext#getFileInfo swallows all IOEs, based on assumption that any and all IOEs means FNF which probably isn’t the intention during rpc exceptions.

       Comment-13)

      StoragePolicySatisfier
      It appears to make back-to-back calls to hasLowRedundancyBlocks and getFileInfo for every file. Haven’t fully groked the code, but if low redundancy is not the common case, then it shouldn’t be called unless/until needed. It looks like files that are under replicated are re-queued again?

      Attachments

        1. HDFS-13110-HDFS-10285-00.patch
          86 kB
          Rakesh Radhakrishnan
        2. HDFS-13110-HDFS-10285-01.patch
          86 kB
          Rakesh Radhakrishnan
        3. HDFS-13110-HDFS-10285-02.patch
          86 kB
          Rakesh Radhakrishnan
        4. HDFS-13110-HDFS-10285-03.patch
          91 kB
          Rakesh Radhakrishnan
        5. HDFS-13110-HDFS-10285-04.patch
          100 kB
          Rakesh Radhakrishnan
        6. HDFS-13110-HDFS-10285-05.patch
          117 kB
          Rakesh Radhakrishnan

        Activity

          People

            rakeshr Rakesh Radhakrishnan
            rakeshr Rakesh Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: