Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5231

S3 build fails because memory estimates changes

    Details

    • Epic Color:
      ghx-label-4

      Description

      The S3 build started to fail due to changed memory estimates.

       TestExplain.test_explain_level0[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test-s3/repos/Impala/bin/../infra/python/env/bin/python
      metadata/test_explain.py:54: in test_explain_level0
          self.run_test_case('QueryTest/explain-level0', vector)
      common/impala_test_suite.py:388: in run_test_case
          self.__verify_results_and_errors(vector, test_section, result, use_db)
      common/impala_test_suite.py:273: in __verify_results_and_errors
          replace_filenames_with_placeholder)
      common/test_result_verifier.py:404: in verify_raw_results
          VERIFIER_MAP[verifier](expected, actual)
      common/test_result_verifier.py:231: in verify_query_result_is_equal
          assert expected_results == actual_results
      E   assert Comparing QueryTestResults (expected vs actual):
      E     'Per-Host Resource Reservation: Memory=136.00MB' == 'Per-Host Resource Reservation: Memory=136.00MB'
      E     'Per-Host Resource Estimates: Memory=388.41MB' != 'Per-Host Resource Estimates: Memory=340.41MB'
      E     '' == ''
      E     'PLAN-ROOT SINK' == 'PLAN-ROOT SINK'
      E     '04:EXCHANGE [UNPARTITIONED]' == '04:EXCHANGE [UNPARTITIONED]'
      E     '02:HASH JOIN [INNER JOIN, BROADCAST]' == '02:HASH JOIN [INNER JOIN, BROADCAST]'
      E     '|--03:EXCHANGE [BROADCAST]' == '|--03:EXCHANGE [BROADCAST]'
      E     '|  01:SCAN HDFS [tpch.orders]' == '|  01:SCAN HDFS [tpch.orders]'
      E     '00:SCAN HDFS [tpch.lineitem]' == '00:SCAN HDFS [tpch.lineitem]'
      

        Issue Links

          Activity

          Hide
          tarasbob Taras Bobrovytsky added a comment -

          Tim, assigning to you because I think this is most likely due to one of your recent patches.

          Show
          tarasbob Taras Bobrovytsky added a comment - Tim, assigning to you because I think this is most likely due to one of your recent patches.
          Hide
          tarmstrong Tim Armstrong added a comment - - edited

          This test was new. Looks like the scan ranges are somehow different for S3. Here's the failing test at a higher explain level:

          08:18:39 E     'Per-Host Resource Reservation: Memory=136.00MB' == 'Per-Host Resource Reservation: Memory=136.00MB'
          08:18:39 E     'Per-Host Resource Estimates: Memory=388.41MB' != 'Per-Host Resource Estimates: Memory=340.41MB'
          08:18:39 E     '' == ''
          08:18:39 E     'F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1' == 'F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1'
          08:18:39 E     'PLAN-ROOT SINK' == 'PLAN-ROOT SINK'
          08:18:39 E     '|  mem-estimate=0B mem-reservation=0B' == '|  mem-estimate=0B mem-reservation=0B'
          08:18:39 E     '|' == '|'
          08:18:39 E     '04:EXCHANGE [UNPARTITIONED]' == '04:EXCHANGE [UNPARTITIONED]'
          08:18:39 E     '|  mem-estimate=0B mem-reservation=0B' == '|  mem-estimate=0B mem-reservation=0B'
          08:18:39 E     '|  tuple-ids=0,1 row-size=454B cardinality=5757710' == '|  tuple-ids=0,1 row-size=454B cardinality=5757710'
          08:18:39 E     '|' == '|'
          08:18:39 E     'F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3' == 'F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3'
          08:18:39 E     '02:HASH JOIN [INNER JOIN, BROADCAST]' == '02:HASH JOIN [INNER JOIN, BROADCAST]'
          08:18:39 E     '|  hash predicates: l_orderkey = o_orderkey' == '|  hash predicates: l_orderkey = o_orderkey'
          08:18:39 E     '|  runtime filters: RF000 <- o_orderkey' == '|  runtime filters: RF000 <- o_orderkey'
          08:18:39 E     '|  mem-estimate=300.41MB mem-reservation=136.00MB' == '|  mem-estimate=300.41MB mem-reservation=136.00MB'
          08:18:39 E     '|  tuple-ids=0,1 row-size=454B cardinality=5757710' == '|  tuple-ids=0,1 row-size=454B cardinality=5757710'
          08:18:39 E     '|' == '|'
          08:18:39 E     '|--03:EXCHANGE [BROADCAST]' == '|--03:EXCHANGE [BROADCAST]'
          08:18:39 E     '|  |  mem-estimate=0B mem-reservation=0B' == '|  |  mem-estimate=0B mem-reservation=0B'
          08:18:39 E     '|  |  tuple-ids=1 row-size=191B cardinality=1500000' == '|  |  tuple-ids=1 row-size=191B cardinality=1500000'
          08:18:39 E     '|  |' == '|  |'
          08:18:39 E     '|  F01:PLAN FRAGMENT [RANDOM] hosts=2 instances=2' != '|  F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3'
          08:18:39 E     '|  01:SCAN HDFS [tpch.orders, RANDOM]' == '|  01:SCAN HDFS [tpch.orders, RANDOM]'
          08:18:39 E     row_regex:.*partitions=1/1 files=1 size=.* == '|     partitions=1/1 files=1 size=162.56MB'
          08:18:39 E     '|     table stats: 1500000 rows total' == '|     table stats: 1500000 rows total'
          08:18:39 E     '|     column stats: all' == '|     column stats: all'
          08:18:39 E     '|     mem-estimate=88.00MB mem-reservation=0B' != '|     mem-estimate=40.00MB mem-reservation=0B'
          08:18:39 E     '|     tuple-ids=1 row-size=191B cardinality=1500000' == '|     tuple-ids=1 row-size=191B cardinality=1500000'
          08:18:39 E     '|' == '|'
          08:18:39 E     '00:SCAN HDFS [tpch.lineitem, RANDOM]' == '00:SCAN HDFS [tpch.lineitem, RANDOM]'
          08:18:39 E     row_regex:.*partitions=1/1 files=1 size=.* == '   partitions=1/1 files=1 size=718.94MB'
          08:18:39 E     '   runtime filters: RF000 -> l_orderkey' == '   runtime filters: RF000 -> l_orderkey'
          08:18:39 E     '   table stats: 6001215 rows total' == '   table stats: 6001215 rows total'
          08:18:39 E     '   column stats: all' == '   column stats: all'
          08:18:39 E     '   mem-estimate=88.00MB mem-reservation=0B' != '   mem-estimate=40.00MB mem-reservation=0B'
          08:18:39 E     '   tuple-ids=0 row-size=263B cardinality=6001215' == '   tuple-ids=0 row-size=263B cardinality=6001215'
          
          Show
          tarmstrong Tim Armstrong added a comment - - edited This test was new. Looks like the scan ranges are somehow different for S3. Here's the failing test at a higher explain level: 08:18:39 E 'Per-Host Resource Reservation: Memory=136.00MB' == 'Per-Host Resource Reservation: Memory=136.00MB' 08:18:39 E 'Per-Host Resource Estimates: Memory=388.41MB' != 'Per-Host Resource Estimates: Memory=340.41MB' 08:18:39 E '' == '' 08:18:39 E 'F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1' == 'F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1' 08:18:39 E 'PLAN-ROOT SINK' == 'PLAN-ROOT SINK' 08:18:39 E '| mem-estimate=0B mem-reservation=0B' == '| mem-estimate=0B mem-reservation=0B' 08:18:39 E '|' == '|' 08:18:39 E '04:EXCHANGE [UNPARTITIONED]' == '04:EXCHANGE [UNPARTITIONED]' 08:18:39 E '| mem-estimate=0B mem-reservation=0B' == '| mem-estimate=0B mem-reservation=0B' 08:18:39 E '| tuple-ids=0,1 row-size=454B cardinality=5757710' == '| tuple-ids=0,1 row-size=454B cardinality=5757710' 08:18:39 E '|' == '|' 08:18:39 E 'F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3' == 'F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3' 08:18:39 E '02:HASH JOIN [INNER JOIN, BROADCAST]' == '02:HASH JOIN [INNER JOIN, BROADCAST]' 08:18:39 E '| hash predicates: l_orderkey = o_orderkey' == '| hash predicates: l_orderkey = o_orderkey' 08:18:39 E '| runtime filters: RF000 <- o_orderkey' == '| runtime filters: RF000 <- o_orderkey' 08:18:39 E '| mem-estimate=300.41MB mem-reservation=136.00MB' == '| mem-estimate=300.41MB mem-reservation=136.00MB' 08:18:39 E '| tuple-ids=0,1 row-size=454B cardinality=5757710' == '| tuple-ids=0,1 row-size=454B cardinality=5757710' 08:18:39 E '|' == '|' 08:18:39 E '|--03:EXCHANGE [BROADCAST]' == '|--03:EXCHANGE [BROADCAST]' 08:18:39 E '| | mem-estimate=0B mem-reservation=0B' == '| | mem-estimate=0B mem-reservation=0B' 08:18:39 E '| | tuple-ids=1 row-size=191B cardinality=1500000' == '| | tuple-ids=1 row-size=191B cardinality=1500000' 08:18:39 E '| |' == '| |' 08:18:39 E '| F01:PLAN FRAGMENT [RANDOM] hosts=2 instances=2' != '| F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3' 08:18:39 E '| 01:SCAN HDFS [tpch.orders, RANDOM]' == '| 01:SCAN HDFS [tpch.orders, RANDOM]' 08:18:39 E row_regex:.*partitions=1/1 files=1 size=.* == '| partitions=1/1 files=1 size=162.56MB' 08:18:39 E '| table stats: 1500000 rows total' == '| table stats: 1500000 rows total' 08:18:39 E '| column stats: all' == '| column stats: all' 08:18:39 E '| mem-estimate=88.00MB mem-reservation=0B' != '| mem-estimate=40.00MB mem-reservation=0B' 08:18:39 E '| tuple-ids=1 row-size=191B cardinality=1500000' == '| tuple-ids=1 row-size=191B cardinality=1500000' 08:18:39 E '|' == '|' 08:18:39 E '00:SCAN HDFS [tpch.lineitem, RANDOM]' == '00:SCAN HDFS [tpch.lineitem, RANDOM]' 08:18:39 E row_regex:.*partitions=1/1 files=1 size=.* == ' partitions=1/1 files=1 size=718.94MB' 08:18:39 E ' runtime filters: RF000 -> l_orderkey' == ' runtime filters: RF000 -> l_orderkey' 08:18:39 E ' table stats: 6001215 rows total' == ' table stats: 6001215 rows total' 08:18:39 E ' column stats: all' == ' column stats: all' 08:18:39 E ' mem-estimate=88.00MB mem-reservation=0B' != ' mem-estimate=40.00MB mem-reservation=0B' 08:18:39 E ' tuple-ids=0 row-size=263B cardinality=6001215' == ' tuple-ids=0 row-size=263B cardinality=6001215'
          Hide
          tarmstrong Tim Armstrong added a comment -

          IMPALA-5231: skip test_explain_level on non-HDFS systems

          Some details of the plans change if we're not running against a 3-node
          minicluster. The point of these tests is to avoid unintended changes
          to the explain format, so we don't need to run it against all FSes.

          Change-Id: I604f83695e956ef6bc85b5d1bc754ccb1378eda1
          Reviewed-on: http://gerrit.cloudera.org:8080/6703
          Reviewed-by: Matthew Jacobs <mj@cloudera.com>
          Tested-by: Impala Public Jenkins

          Show
          tarmstrong Tim Armstrong added a comment - IMPALA-5231 : skip test_explain_level on non-HDFS systems Some details of the plans change if we're not running against a 3-node minicluster. The point of these tests is to avoid unintended changes to the explain format, so we don't need to run it against all FSes. Change-Id: I604f83695e956ef6bc85b5d1bc754ccb1378eda1 Reviewed-on: http://gerrit.cloudera.org:8080/6703 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Impala Public Jenkins —

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarasbob Taras Bobrovytsky
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development