Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2861

5.4.x-nightly-data-load: test_scanners: Memory Limit Exceeded

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2.10
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
      None

      Description

      http://sandbox.jenkins.cloudera.com/job/impala-cdh5-2.2.0_5.4.x-nightly-data-load/189/testReport/junit/query_test.test_scanners/TestScannersAllTableFormats/test_scanners_batch_size__1___exec_option____disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0___batch_size___0___num_nodes___0____table_format__parquet_none_/

      E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      E    Query aborted:
      E   Memory limit exceeded
      E   
      E   
      E   
      E   Memory Limit Exceeded
      E   Query(5b4e5d9da153dea4:1985436140f29e93) Limit: memory limit exceeded. Limit=5.00 GB Consumption=5.00 GB
      E     Fragment 5b4e5d9da153dea4:1985436140f29e97: Consumption=5.00 GB
      E       SORT_NODE (id=5): Consumption=4.00 KB
      E       CROSS_JOIN_NODE (id=4): Consumption=5.00 GB
      E       CROSS_JOIN_NODE (id=3): Consumption=16.00 KB
      E       HDFS_SCAN_NODE (id=2): Consumption=0
      E       EXCHANGE_NODE (id=6): Consumption=0
      E       DataStreamRecvr: Consumption=0
      E       EXCHANGE_NODE (id=7): Consumption=0
      E       DataStreamRecvr: Consumption=3.32 MB
      E       DataStreamSender: Consumption=4.00 KB
      E     Block Manager: Limit=4.00 GB Consumption=0
      Standard Error
      
      -- connecting to: localhost:21000
      -- executing against localhost:21000
      use functional_parquet;
      
      SET disable_codegen=False;
      SET abort_on_error=1;
      SET exec_single_node_rows_threshold=0;
      SET batch_size=1;
      SET num_nodes=0;
      -- executing against localhost:21000
      select count(*),
        sum(id), count(bool_col), sum(tinyint_col), sum(smallint_col),
        sum(int_col), sum(bigint_col), max(float_col), max(double_col),
        max(date_string_col), max(string_col), max(timestamp_col)
      from alltypesagg
      where id % 2 = 0 and day is not null;
      
      -- executing against localhost:21000
      select sum(t1.id), sum(t1.int_col),max(t1.date_string_col), max(t2.string_col)
      from alltypesagg t1
      inner join alltypesagg t2
        on t1.id = t2.id and t1.day is not null and t2.day is not null;
      
      -- executing against localhost:21000
      select id, bool_col, int_col
      from alltypesagg where day is not null
      order by 1 desc, 2 desc, 3 desc
      limit 10;
      

      Started failing after this backport

      IMPALA-2473: reduce scanner memory usage
      

        Attachments

          Activity

            People

            • Assignee:
              jyu@cloudera.com Juan Yu
              Reporter:
              dhecht Daniel Hecht
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: