Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8486

test_udf_update_via_drop and test_udf_update_via_create fail on local catalog

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 3.3.0
    • Impala 3.3.0
    • Catalog
    • ghx-label-2

    Description

       TestUdfTargeted.test_udf_update_via_drop[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none] 
      tests/query_test/test_udfs.py:541: in test_udf_update_via_drop
          self._run_query_all_impalads(exec_options, query_stmt, ["New UDF"])
      tests/query_test/test_udfs.py:52: in _run_query_all_impalads
          assert result.data == expected
      E   assert ['Old UDF'] == ['New UDF']
      E     At index 0 diff: 'Old UDF' != 'New UDF'
      E     Full diff:
      E     - ['Old UDF']
      E     + ['New UDF']
      ----------------------------
      

      The tests are checking that the local UDF caches on each impalad get invalidated by a drop/create of a function referencing the HDFS file containing the UDF. The test fails because the local catalog, unlike the regular catalog, doesn't invalidate LibCache entries upon receiving a catalog update.

      I looked at this for long enough to realise that the invalidation mechanism is fundamentally broken - it doesn't work with dedicated executors. It also creates a race between the statestore updates and queries referencing the UDFs - if the queries win the race, then they can incorrectly use the old version that should have been invalidated.

      I think this is a potentially problematic issue because old JAR/SO versions could persist in the cache indefinitely if old versions are overwritten in place.

      Attachments

        Issue Links

          Activity

            People

              stigahuang Quanlong Huang
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: