Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9272

PlannerTest.testHdfs fails due to dependence on result of year(now())

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • Impala 3.4.0
    • None
    • ghx-label-11

    Description

      PlannerTest.testHdfs fails after we enter the new year(2020). It assumes that the result of year(now()) is 2019 but it's 2020 now. We should not use year(now()) in these tests.

      Section PLAN of query:
      select * from functional.alltypesagg t1
      where t1.day = instr("this is a test", "this") or t1.year = year(now()) + 100
      
      Actual does not match expected result:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` = 1 OR t1.`year` = 2120
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         HDFS partitions=1/11 files=1 size=73.39KB
         row-size=95B cardinality=1.00K
      
      Expected:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` = 1 OR t1.`year` = 2119
         partitions=1/11 files=1 size=73.39KB
         row-size=95B cardinality=1.00K
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      Per-Host Resources: mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=2
        PLAN-ROOT SINK
        |  output exprs: t1.id, t1.bool_col, t1.tinyint_col, t1.smallint_col, t1.int_col, t1.bigint_col, t1.float_col, t1.double_col, t1.date_string_col, t1.string_col, t1.timestamp_col, t1.year, t1.month, t1.day
        |  mem-estimate=0B mem-reservation=0B thread-reservation=0
        |
        00:SCAN HDFS [functional.alltypesagg t1]
           partition predicates: t1.`day` = CAST(1 AS INT) OR t1.`year` = CAST(2120 AS INT)
           HDFS partitions=1/11 files=1 size=73.39KB
           stored statistics:
             table: rows=11.00K size=814.73KB
             partitions: 1/1 rows=1.00K
             columns: all
           extrapolated-rows=disabled max-scan-range-rows=11.00K
           mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=1
           tuple-ids=0 row-size=95B cardinality=1.00K
           in pipelines: 00(GETNEXT)
      
      Section PLAN of query:
      select * from functional.alltypesagg t1
      where t1.day in (1, cast(2.0 as INT), year(now()) + 100)
      
      Actual does not match expected result:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` IN (1, 2, 2120)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         HDFS partitions=2/11 files=2 size=147.87KB
         row-size=95B cardinality=2.00K
      
      Expected:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` IN (1, 2, 2119)
         partitions=2/11 files=2 size=147.87KB
         row-size=95B cardinality=2.00K
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      Per-Host Resources: mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=2
        PLAN-ROOT SINK
        |  output exprs: t1.id, t1.bool_col, t1.tinyint_col, t1.smallint_col, t1.int_col, t1.bigint_col, t1.float_col, t1.double_col, t1.date_string_col, t1.string_col, t1.timestamp_col, t1.year, t1.month, t1.day
        |  mem-estimate=0B mem-reservation=0B thread-reservation=0
        |
        00:SCAN HDFS [functional.alltypesagg t1]
           partition predicates: t1.`day` IN (CAST(1 AS INT), CAST(2 AS INT), CAST(2120 AS INT))
           HDFS partitions=2/11 files=2 size=147.87KB
           stored statistics:
             table: rows=11.00K size=814.73KB
             partitions: 2/2 rows=2.00K
             columns: all
           extrapolated-rows=disabled max-scan-range-rows=5.54K
           mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=1
           tuple-ids=0 row-size=95B cardinality=2.00K
           in pipelines: 00(GETNEXT)
      Stacktracejava.lang.AssertionError: 
      
      Section PLAN of query:
      select * from functional.alltypesagg t1
      where t1.day = instr("this is a test", "this") or t1.year = year(now()) + 100
      
      Actual does not match expected result:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` = 1 OR t1.`year` = 2120
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         HDFS partitions=1/11 files=1 size=73.39KB
         row-size=95B cardinality=1.00K
      
      Expected:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` = 1 OR t1.`year` = 2119
         partitions=1/11 files=1 size=73.39KB
         row-size=95B cardinality=1.00K
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      Per-Host Resources: mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=2
        PLAN-ROOT SINK
        |  output exprs: t1.id, t1.bool_col, t1.tinyint_col, t1.smallint_col, t1.int_col, t1.bigint_col, t1.float_col, t1.double_col, t1.date_string_col, t1.string_col, t1.timestamp_col, t1.year, t1.month, t1.day
        |  mem-estimate=0B mem-reservation=0B thread-reservation=0
        |
        00:SCAN HDFS [functional.alltypesagg t1]
           partition predicates: t1.`day` = CAST(1 AS INT) OR t1.`year` = CAST(2120 AS INT)
           HDFS partitions=1/11 files=1 size=73.39KB
           stored statistics:
             table: rows=11.00K size=814.73KB
             partitions: 1/1 rows=1.00K
             columns: all
           extrapolated-rows=disabled max-scan-range-rows=11.00K
           mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=1
           tuple-ids=0 row-size=95B cardinality=1.00K
           in pipelines: 00(GETNEXT)
      
      Section PLAN of query:
      select * from functional.alltypesagg t1
      where t1.day in (1, cast(2.0 as INT), year(now()) + 100)
      
      Actual does not match expected result:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` IN (1, 2, 2120)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         HDFS partitions=2/11 files=2 size=147.87KB
         row-size=95B cardinality=2.00K
      
      Expected:
      PLAN-ROOT SINK
      |
      00:SCAN HDFS [functional.alltypesagg t1]
         partition predicates: t1.`day` IN (1, 2, 2119)
         partitions=2/11 files=2 size=147.87KB
         row-size=95B cardinality=2.00K
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      Per-Host Resources: mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=2
        PLAN-ROOT SINK
        |  output exprs: t1.id, t1.bool_col, t1.tinyint_col, t1.smallint_col, t1.int_col, t1.bigint_col, t1.float_col, t1.double_col, t1.date_string_col, t1.string_col, t1.timestamp_col, t1.year, t1.month, t1.day
        |  mem-estimate=0B mem-reservation=0B thread-reservation=0
        |
        00:SCAN HDFS [functional.alltypesagg t1]
           partition predicates: t1.`day` IN (CAST(1 AS INT), CAST(2 AS INT), CAST(2120 AS INT))
           HDFS partitions=2/11 files=2 size=147.87KB
           stored statistics:
             table: rows=11.00K size=814.73KB
             partitions: 2/2 rows=2.00K
             columns: all
           extrapolated-rows=disabled max-scan-range-rows=5.54K
           mem-estimate=32.00MB mem-reservation=128.00KB thread-reservation=1
           tuple-ids=0 row-size=95B cardinality=2.00K
           in pipelines: 00(GETNEXT)
      
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:924)
      	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:929)
      	at org.apache.impala.planner.PlannerTest.testHdfs(PlannerTest.java:307)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)

      Jenkins job: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9453

      Attachments

        Activity

          People

            stigahuang Quanlong Huang
            stigahuang Quanlong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: