Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16145 Replicate functions
  3. HIVE-16591

DR for function Binaries on HDFS

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • HiveServer2
    • None

    Description

      1. We have to make sure that during incremental dump we dont allow functions to be copied if they have local filesystem "file://" resources. – depends how much system side work we want to do, We are going to explicitly provide a caveat for replicating functions where in, only functions created "using" clause will be replicated and the "using" clause prohibits creating functions with the local "file://" resources and hence doing additional checks when doing repl dump might not be required.
      2. We have to make sure that during the bootstrap / incremental dump we append the namenode host + port if functions are created without the fully qualified location of uri on hdfs, not sure how this would play for S3 or WASB filesystem.
      3. We have to copy the binaries of a function resource list on CREATE / DROP FUNCTION . The change management file system has to keep a copy of the binary when a DROP function is called, to provide capability of updating binary definition for existing functions along with DR. An example of list of steps is given in doc (ReplicateFunctions.pdf ) attached in parent Issue.

      Attachments

        1. HIVE-16591.1.patch
          70 kB
          Anishek Agarwal
        2. HIVE-16591.2.patch
          70 kB
          Anishek Agarwal
        3. HIVE-16591.3.patch
          70 kB
          Anishek Agarwal

        Issue Links

          Activity

            People

              anishek Anishek Agarwal
              anishek Anishek Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: