Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12852 LLAP: whitelist-based UDF support
  3. HIVE-12856

LLAP: update (add/remove) the UDFs available in LLAP when they are changed (refresh periodically)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      I don't think re-querying the functions is going to scale, and the sessions obviously cannot notify all LLAP clusters of every change. We should add global versioning to metastore functions to track changes, and then possibly add a notification mechanism, potentially thru ZK to avoid overloading the metastore itself.

      Attachments

        1. HIVE-12856.01.nogen.patch
          29 kB
          Sergey Shelukhin
        2. HIVE-12856.01.patch
          58 kB
          Sergey Shelukhin
        3. HIVE-12856.02.nogen.patch
          31 kB
          Sergey Shelukhin
        4. HIVE-12856.02.patch
          59 kB
          Sergey Shelukhin
        5. HIVE-12856.nogen.patch
          29 kB
          Sergey Shelukhin
        6. HIVE-12856.patch
          544 kB
          Sergey Shelukhin

        Issue Links

          blocks

          Sub-task - The sub-task of the issue HIVE-13006 LLAP: add finer-grained classloaders as an option to be able to block the usage of removed UDFs

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue HIVE-13007 add an API to force reload UDFs to LLAP (either force reload everything, or force a regular refresh)

          • Major - Major loss of function.
          • Open
          is blocked by

          Sub-task - The sub-task of the issue HIVE-12892 Add global change versioning to permanent functions in metastore

          • Major - Major loss of function.
          • Reopened

          Sub-task - The sub-task of the issue HIVE-12918 LLAP should never create embedded metastore when localizing functions

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB

          Activity

            People

              sershe Sergey Shelukhin
              sershe Sergey Shelukhin
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: