Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7023

TestInsertQueries.test_insert_overwrite fails by hitting memory limit

    XMLWordPrintableJSON

Details

    • ghx-label-5

    Description

      This failure is seen on exhaustive builds on both master and 2.x:

      Error Message
      ImpalaBeeswaxException: ImpalaBeeswaxException:  INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>  MESSAGE: AnalysisException: Failed to evaluate expr: 20 CAUSED BY: InternalException: Memory limit exceeded: Error occurred on backend impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by fragment 0:0 Memory left in process limit: -4.29 GB Query(d24ea53242b4cedc:fc8e088500000000): Total=0 Peak=0   <unnamed>: Total=0 Peak=0Process: memory limit exceeded. Limit=12.00 GB Total=16.29 GB Peak=16.29 GB   Buffer Pool: Free Buffers: Total=160.00 KB   Buffer Pool: Clean Pages: Total=0   Buffer Pool: Unused Reservation: Total=-328.00 KB   Data Stream Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB   Data Stream Manager Early RPCs: Total=0 Peak=6.76 KB   TCMalloc Overhead: Total=103.56 MB   RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB     Query(d24ea53242b4cedc:fc8e088500000000): Total=0 Peak=0   RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB     Query(f4014f7bb49ea78:6b926b1900000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB Total=1.76 GB Peak=1.96 GB     Query(2c44b65fbcb4e1ce:3d73badf00000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.04 GB Total=1.04 GB Peak=1.61 GB     Query(214cc23c1376176f:7844977b00000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB Total=1.23 GB Peak=1.23 GB     Query(8949bdf792a32ad2:33a36c0300000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=642.20 MB Total=642.20 MB Peak=642.20 MB     Query(5412ff4e6065721:519d3e6100000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=556.49 MB Total=556.49 MB Peak=556.49 MB   Untracked Memory: Total=10.98 GB
      Stacktrace
      query_test/test_insert.py:132: in test_insert_overwrite
          multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
      common/impala_test_suite.py:405: in run_test_case
          result = self.__execute_query(target_impalad_client, query, user=user)
      common/impala_test_suite.py:620: in __execute_query
          return impalad_client.execute(query, user=user)
      common/impala_connection.py:160: in execute
          return self.__beeswax_client.execute(sql_stmt, user=user)
      beeswax/impala_beeswax.py:173: in execute
          handle = self.__execute_query(query_string.strip(), user=user)
      beeswax/impala_beeswax.py:339: in __execute_query
          handle = self.execute_query_async(query_string, user=user)
      beeswax/impala_beeswax.py:335: in execute_query_async
          return self.__do_rpc(lambda: self.imp_service.query(query,))
      beeswax/impala_beeswax.py:460: in __do_rpc
          raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      E    MESSAGE: AnalysisException: Failed to evaluate expr: 20
      E   CAUSED BY: InternalException: Memory limit exceeded: Error occurred on backend impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by fragment 0:0
      E   Memory left in process limit: -4.29 GB
      E   Query(d24ea53242b4cedc:fc8e088500000000): Total=0 Peak=0
      E     <unnamed>: Total=0 Peak=0Process: memory limit exceeded. Limit=12.00 GB Total=16.29 GB Peak=16.29 GB
      E     Buffer Pool: Free Buffers: Total=160.00 KB
      E     Buffer Pool: Clean Pages: Total=0
      E     Buffer Pool: Unused Reservation: Total=-328.00 KB
      E     Data Stream Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB
      E     Data Stream Manager Early RPCs: Total=0 Peak=6.76 KB
      E     TCMalloc Overhead: Total=103.56 MB
      E     RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB
      E       Query(d24ea53242b4cedc:fc8e088500000000): Total=0 Peak=0
      E     RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB
      E       Query(f4014f7bb49ea78:6b926b1900000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB Total=1.76 GB Peak=1.96 GB
      E       Query(2c44b65fbcb4e1ce:3d73badf00000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.04 GB Total=1.04 GB Peak=1.61 GB
      E       Query(214cc23c1376176f:7844977b00000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB Total=1.23 GB Peak=1.23 GB
      E       Query(8949bdf792a32ad2:33a36c0300000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=642.20 MB Total=642.20 MB Peak=642.20 MB
      E       Query(5412ff4e6065721:519d3e6100000000): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=556.49 MB Total=556.49 MB Peak=556.49 MB
      E     Untracked Memory: Total=10.98 GB

      Attachments

        Activity

          People

            joemcdonnell Joe McDonnell
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: