Hive
  1. Hive
  2. HIVE-1948

Have audit logging in the Metastore

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Logging, Metastore, Security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).

      1. audit-log-3.patch
        6 kB
        Devaraj Das
      2. audit-log-2.patch
        7 kB
        Devaraj Das
      3. audit-log.patch
        5 kB
        Devaraj Das
      4. audit-log.1.patch
        5 kB
        Devaraj Das

        Issue Links

          Activity

          Hide
          Namit Jain added a comment -

          Committed. Thanks Devaraj

          Show
          Namit Jain added a comment - Committed. Thanks Devaraj
          Hide
          Namit Jain added a comment -

          +1

          Show
          Namit Jain added a comment - +1
          Hide
          Devaraj Das added a comment -

          Regenerated patch

          Show
          Devaraj Das added a comment - Regenerated patch
          Hide
          Namit Jain added a comment -

          Can you regenerate the patch - I am getting some merge conflicts.

          Show
          Namit Jain added a comment - Can you regenerate the patch - I am getting some merge conflicts.
          Hide
          Devaraj Das added a comment -

          In this patch I made the logging to happen only when a secure shim is deployed (the case where such audit logging makes most sense).

          Show
          Devaraj Das added a comment - In this patch I made the logging to happen only when a secure shim is deployed (the case where such audit logging makes most sense).
          Hide
          Namit Jain added a comment -

          What about the performance impact for this ?
          I mean, there seems to be no way to turn it off - is reading the conf.

          try

          { ugi = ShimLoader.getHadoopShims().getUGIForConf(getConf()); }

          catch (Exception ex)

          { throw new RuntimeException(ex); }

          for every audit operation acceptable ?

          Show
          Namit Jain added a comment - What about the performance impact for this ? I mean, there seems to be no way to turn it off - is reading the conf. try { ugi = ShimLoader.getHadoopShims().getUGIForConf(getConf()); } catch (Exception ex) { throw new RuntimeException(ex); } for every audit operation acceptable ?
          Hide
          Devaraj Das added a comment -

          https://reviews.apache.org/r/398/ is the reviewboard URL

          Show
          Devaraj Das added a comment - https://reviews.apache.org/r/398/ is the reviewboard URL
          Hide
          Devaraj Das added a comment -

          Submitting patch for review. There is one caveat with this patch - it won't log the IP address of the remote clients when security is enabled in Hive. Making this work means a change in thrift. I have raised THRIFT-1053 for the same. Once THRIFT-1053 is addressed, I will provide a fix (in a different jira) to capture the IP address for the secure case too.

          Show
          Devaraj Das added a comment - Submitting patch for review. There is one caveat with this patch - it won't log the IP address of the remote clients when security is enabled in Hive. Making this work means a change in thrift. I have raised THRIFT-1053 for the same. Once THRIFT-1053 is addressed, I will provide a fix (in a different jira) to capture the IP address for the secure case too.
          Hide
          Devaraj Das added a comment -

          A slightly updated patch.

          Show
          Devaraj Das added a comment - A slightly updated patch.
          Hide
          Devaraj Das added a comment -

          Early patch. I am testing it.

          Show
          Devaraj Das added a comment - Early patch. I am testing it.

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development