Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7699

TestSpillingNoDebugActionDimensions fails earlier than expected

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • Impala 3.1.0
    • None
    • ghx-label-5

    Description

      In some of the recent runs, the query fails without sufficient memory, however it is in the HDFS scan node rather than the hash join node. Here are the corresponding logs: 

      Stacktrace:

      query_test/test_spilling.py:113: in test_spilling_no_debug_action self.run_test_case('QueryTest/spilling-no-debug-action', vector) common/impala_test_suite.py:466: in run_test_case self.__verify_exceptions(test_section['CATCH'], str(e), use_db) common/impala_test_suite.py:319: in __verify_exceptions (expected_str, actual_str) E AssertionError: Unexpected exception string. Expected: row_regex:.*Cannot perform hash join at node with id .*. Repartitioning did not reduce the size of a spilled partition.* E Not found in actual: ImpalaBeeswaxException: Query aborted:Memory limit exceeded: Failed to allocate tuple bufferHDFS_SCAN_NODE (id=1) could not allocate 190.00 KB without exceeding limit.Error occurred on backend localhost:22001 by fragment 2e4f0f944d373848:9ae1d7e200000002
      

       

      Here are the impalad logs: 

      I1010 18:31:30.721693  7270 coordinator.cc:498] ExecState: query id=2e4f0f944d373848:9ae1d7e200000000 finstance=2e4f0f944d373848:9ae1d7e200000002 on host=localhost:22001 (EXECUTING -> ERROR) status=Memory limit exceeded: Failed to allocate tuple buffer
      HDFS_SCAN_NODE (id=1) could not allocate 190.00 KB without exceeding limit.
      Error occurred on backend localhost:22001 by fragment 2e4f0f944d373848:9ae1d7e200000002
      Memory left in process limit: 9.19 GB
      Memory left in query limit: 157.62 KB
      Query(2e4f0f944d373848:9ae1d7e200000000): Limit=150.00 MB Reservation=117.25 MB ReservationLimit=118.00 MB OtherMemory=32.60 MB Total=149.85 MB Peak=149.85 MB
        Unclaimed reservations: Reservation=5.75 MB OtherMemory=0 Total=5.75 MB Peak=55.75 MB
        Fragment 2e4f0f944d373848:9ae1d7e200000003: Reservation=2.00 MB OtherMemory=22.20 MB Total=24.20 MB Peak=24.20 MB
          Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
          SORT_NODE (id=3): Total=0 Peak=0
          HASH_JOIN_NODE (id=2): Total=42.25 KB Peak=42.25 KB
            Exprs: Total=13.12 KB Peak=13.12 KB
            Hash Join Builder (join_node_id=2): Total=13.12 KB Peak=13.12 KB
              Hash Join Builder (join_node_id=2) Exprs: Total=13.12 KB Peak=13.12 KB
          HDFS_SCAN_NODE (id=0): Total=0 Peak=0
          EXCHANGE_NODE (id=4): Reservation=18.79 MB OtherMemory=235.89 KB Total=19.02 MB Peak=19.02 MB
            KrpcDeferredRpcs: Total=235.89 KB Peak=235.89 KB
          KrpcDataStreamSender (dst_id=5): Total=480.00 B Peak=480.00 B
          CodeGen: Total=3.13 MB Peak=3.13 MB
        Fragment 2e4f0f944d373848:9ae1d7e200000002: Reservation=109.50 MB OtherMemory=10.39 MB Total=119.89 MB Peak=119.89 MB
          HDFS_SCAN_NODE (id=1): Reservation=109.50 MB OtherMemory=10.20 MB Total=119.70 MB Peak=119.70 MB
            Queued Batches: Total=6.12 MB Peak=6.12 MB
          KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
          CodeGen: Total=488.00 B Peak=51.00 KB
      

      Attachments

        Activity

          People

            bikramjeet.vig Bikramjeet Vig
            poojanilangekar Pooja Nilangekar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: