Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19870

HCatalog dynamic partition query can fail, if the table path is managed by Sentry

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: None
    • Labels:
      None

      Description

      The central issue is that HCatalog is assuming it needs to handle the storage based authorization features. When a job completes, in HCatalog's file committing phase it tries to manually set the permissions for the table for authorization's sake, which makes it go against auto-authorization managment features provided by Sentry.

      The offending code is specifically at https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L373-L374

      and

      https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L380-L385

      and

      https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L952-L954

      where you can notice that after it moves the files created by the job into their respective partition directories under the final table destination, it goes onto trying to perform chmod/chgrp operations which will fail out

        Attachments

        1. HIVE-19870.patch
          4 kB
          Peter Vary
        2. HIVE-19870.5.patch
          6 kB
          Peter Vary
        3. HIVE-19870.4.patch
          6 kB
          Peter Vary
        4. HIVE-19870.3.patch
          6 kB
          Peter Vary
        5. HIVE-19870.2.patch
          3 kB
          Peter Vary

          Activity

            People

            • Assignee:
              pvary Peter Vary
              Reporter:
              pvary Peter Vary
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: