Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9736

StorageBasedAuthProvider should batch namenode-calls where possible.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2.1
    • Fix Version/s: None
    • Component/s: Metastore, Security
    • Labels:

      Description

      Consider a table partitioned by 2 keys (dt, region). Say a dt partition could have 10000 associated regions. Consider that the user does:

      ALTER TABLE my_table DROP PARTITION (dt='20150101');
      

      As things stand now, StorageBasedAuthProvider will make individual DistributedFileSystem.listStatus() calls for each partition-directory, and authorize each one separately. It'd be faster to batch the calls, and examine multiple FileStatus objects at once.

        Attachments

        1. HIVE-9736.1.patch
          27 kB
          Mithun Radhakrishnan
        2. HIVE-9736.2.patch
          28 kB
          Mithun Radhakrishnan
        3. HIVE-9736.3.patch
          28 kB
          Mithun Radhakrishnan
        4. HIVE-9736.4.patch
          28 kB
          Mithun Radhakrishnan
        5. HIVE-9736.5.patch
          35 kB
          Mithun Radhakrishnan
        6. HIVE-9736.6.patch
          28 kB
          Mithun Radhakrishnan
        7. HIVE-9736.7.patch
          28 kB
          Mithun Radhakrishnan
        8. HIVE-9736.8.patch
          28 kB
          Sushanth Sowmyan

          Issue Links

            Activity

              People

              • Assignee:
                mithun Mithun Radhakrishnan
                Reporter:
                mithun Mithun Radhakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: