Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4342

"Cannot drop non-empty database: udf_test_errors" when using local filesystem

    Details

      Description

      TestUdfs.test_udf_errors[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
      query_test/test_udfs.py:81: in test_udf_errors
          self.run_test_case('QueryTest/udf-errors', vector)
      common/impala_test_suite.py:320: in run_test_case
          result = self.__execute_query(target_impalad_client, query, user=user)
      common/impala_test_suite.py:511: in __execute_query
          return impalad_client.execute(query, user=user)
      common/impala_connection.py:160: in execute
          return self.__beeswax_client.execute(sql_stmt, user=user)
      beeswax/impala_beeswax.py:173: in execute
          handle = self.__execute_query(query_string.strip(), user=user)
      beeswax/impala_beeswax.py:337: in __execute_query
          handle = self.execute_query_async(query_string, user=user)
      beeswax/impala_beeswax.py:333: in execute_query_async
          return self.__do_rpc(lambda: self.imp_service.query(query,))
      beeswax/impala_beeswax.py:458: in __do_rpc
          raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      E    MESSAGE: AnalysisException: Cannot drop non-empty database: udf_test_errors
      

      I think this was from https://gerrit.cloudera.org/#/c/4651 , so I assigned to you, Michael. COuld this be some race condition? I ask because the build with the error was bf1d9677fc09dabc88ab6c71f82ec99ec59ec164
      and the build right before it, which was green, was on 502220c69d483f785632eaf0030babf40be78ff4

        Issue Links

          Activity

          Hide
          kwho Michael Ho added a comment -

          b15d992abe09bc841f6e2112d47099eb15f8454f modifies test_udf_errors.test but as you said, the failure is not deterministic and that particular test shouldn't have any race condition as far as I know so I am doubtful it's caused by that commit.

          I suspect this is a metadata issue similar to IMPALA-3888.

          Show
          kwho Michael Ho added a comment - b15d992abe09bc841f6e2112d47099eb15f8454f modifies test_udf_errors.test but as you said, the failure is not deterministic and that particular test shouldn't have any race condition as far as I know so I am doubtful it's caused by that commit. I suspect this is a metadata issue similar to IMPALA-3888 .
          Hide
          kwho Michael Ho added a comment -

          A change is out for review: https://gerrit.cloudera.org/#/c/4792/ to workaround the issue.

          Show
          kwho Michael Ho added a comment - A change is out for review: https://gerrit.cloudera.org/#/c/4792/ to workaround the issue.
          Hide
          kwho Michael Ho added a comment -

          Has only happened once so far and tried looping the same test locally for 500 times and no repro. Downgrading it for now.

          Show
          kwho Michael Ho added a comment - Has only happened once so far and tried looping the same test locally for 500 times and no repro. Downgrading it for now.
          Show
          lv Lars Volker added a comment - Happened again here: http://sandbox.jenkins.cloudera.com/view/Impala/view/Evergreen-asf-master/job/impala-asf-master-core-non-partitioned-joins-and-aggs/115/console
          Hide
          lv Lars Volker added a comment -

          As discussed in the gerrit change this might be another FE race similar to IMPALA-3641.

          Show
          lv Lars Volker added a comment - As discussed in the gerrit change this might be another FE race similar to IMPALA-3641 .
          Hide
          henryr Henry Robinson added a comment -

          Showed up again (in a downstream build of ours): http://sandbox.jenkins.cloudera.com/job/impala-cdh5-trunk-core-data-load/1108/console

          If it is IMPALA-3641, that's becoming quite a serious bug.

          Show
          henryr Henry Robinson added a comment - Showed up again (in a downstream build of ours): http://sandbox.jenkins.cloudera.com/job/impala-cdh5-trunk-core-data-load/1108/console If it is IMPALA-3641 , that's becoming quite a serious bug.
          Hide
          kwho Michael Ho added a comment -

          Let's track the FE problem in IMPALA-3641.

          Show
          kwho Michael Ho added a comment - Let's track the FE problem in IMPALA-3641 .

            People

            • Assignee:
              kwho Michael Ho
              Reporter:
              jbapple Jim Apple
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development