Details

      Description

      Currently, Impala relies on INVALIDATE METADATA to load externally added UDFs from HMS. However, INVALIDATE METADATA affects all the databases/tables of an Impala cluster. Instead, we should add a REFRESH FUNCTIONS <db> statement to reload the functions of a database from HMS. This statement should return the list of db functions to the issuing Impalad in order to update its local catalog cache.

        Activity

        Hide
        tarasbob Taras Bobrovytsky added a comment -
        commit 24ff0f2fc21f0111f4c4fa9cbab40105e1e7ee01
        Author: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
        Date:   Mon May 8 13:18:10 2017 -0700
        
            IMPALA-5259: Add REFRESH FUNCTIONS <db> statement
            
            Before this patch, Impala relied on INVALIDATE METADATA to load
            externally added UDFs from HMS. The problem with this approach is that
            INVALIDATE METADATA affects all databases and tables in the entire
            cluster.
            
            In this patch, we add a REFRESH FUNCTIONS <db> statement that reloads
            the functions of a database from HMS. We return a list of updated and
            removed db functions to the issuing Impalad in order to update its
            local catalog cache.
            
            Testing:
            - Ran a private build which passed.
            
            Change-Id: I3625c88bb51cca833f3293c224d3f0feb00e6e0b
            Reviewed-on: http://gerrit.cloudera.org:8080/6878
            Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
            Tested-by: Impala Public Jenkins
        
        Show
        tarasbob Taras Bobrovytsky added a comment - commit 24ff0f2fc21f0111f4c4fa9cbab40105e1e7ee01 Author: Taras Bobrovytsky <tbobrovytsky@cloudera.com> Date: Mon May 8 13:18:10 2017 -0700 IMPALA-5259: Add REFRESH FUNCTIONS <db> statement Before this patch, Impala relied on INVALIDATE METADATA to load externally added UDFs from HMS. The problem with this approach is that INVALIDATE METADATA affects all databases and tables in the entire cluster. In this patch, we add a REFRESH FUNCTIONS <db> statement that reloads the functions of a database from HMS. We return a list of updated and removed db functions to the issuing Impalad in order to update its local catalog cache. Testing: - Ran a private build which passed. Change-Id: I3625c88bb51cca833f3293c224d3f0feb00e6e0b Reviewed-on: http: //gerrit.cloudera.org:8080/6878 Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com> Tested-by: Impala Public Jenkins

          People

          • Assignee:
            tarasbob Taras Bobrovytsky
            Reporter:
            dtsirogiannis Dimitris Tsirogiannis
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development