Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9736

StorageBasedAuthProvider should batch namenode-calls where possible.

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 1.2.1
    • None
    • Metastore, Security

    Description

      Consider a table partitioned by 2 keys (dt, region). Say a dt partition could have 10000 associated regions. Consider that the user does:

      ALTER TABLE my_table DROP PARTITION (dt='20150101');
      

      As things stand now, StorageBasedAuthProvider will make individual DistributedFileSystem.listStatus() calls for each partition-directory, and authorize each one separately. It'd be faster to batch the calls, and examine multiple FileStatus objects at once.

      Attachments

        1. HIVE-9736.1.patch
          27 kB
          Mithun Radhakrishnan
        2. HIVE-9736.2.patch
          28 kB
          Mithun Radhakrishnan
        3. HIVE-9736.3.patch
          28 kB
          Mithun Radhakrishnan
        4. HIVE-9736.4.patch
          28 kB
          Mithun Radhakrishnan
        5. HIVE-9736.5.patch
          35 kB
          Mithun Radhakrishnan
        6. HIVE-9736.6.patch
          28 kB
          Mithun Radhakrishnan
        7. HIVE-9736.7.patch
          28 kB
          Mithun Radhakrishnan
        8. HIVE-9736.8.patch
          28 kB
          Sushanth Sowmyan

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mithun Mithun Radhakrishnan Assign to me
            mithun Mithun Radhakrishnan

            Dates

              Created:
              Updated:

              Slack

                Issue deployment