Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-13110

[SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-10285, 3.2.0
    • None
    • None

    Description

      This task is to address the following Daryn Sharp's comments. Please refer HDFS-10285 to see more detailed discussion.

      Comment-10)

      NamenodeProtocolTranslatorPB

      Most of the api changes appear unnecessary.

      IntraSPSNameNodeContext#getFileInfo swallows all IOEs, based on assumption that any and all IOEs means FNF which probably isn’t the intention during rpc exceptions.

       Comment-13)

      StoragePolicySatisfier
      It appears to make back-to-back calls to hasLowRedundancyBlocks and getFileInfo for every file. Haven’t fully groked the code, but if low redundancy is not the common case, then it shouldn’t be called unless/until needed. It looks like files that are under replicated are re-queued again?

      Attachments

        1. HDFS-13110-HDFS-10285-05.patch
          117 kB
          Rakesh Radhakrishnan
        2. HDFS-13110-HDFS-10285-04.patch
          100 kB
          Rakesh Radhakrishnan
        3. HDFS-13110-HDFS-10285-03.patch
          91 kB
          Rakesh Radhakrishnan
        4. HDFS-13110-HDFS-10285-02.patch
          86 kB
          Rakesh Radhakrishnan
        5. HDFS-13110-HDFS-10285-01.patch
          86 kB
          Rakesh Radhakrishnan
        6. HDFS-13110-HDFS-10285-00.patch
          86 kB
          Rakesh Radhakrishnan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rakeshr Rakesh Radhakrishnan Assign to me
            rakeshr Rakesh Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment