Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-777

Partition column in function parameter occurs NPE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0, 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      See the following case

      create table testQueryCasesOnColumnPartitionedTable (col1 int4, col2 int4, null_col int4) partition by column(key float8)
      
      select pow(key, 2) from testQueryCasesOnColumnPartitionedTable
      
      2014-04-19 10:27:54,455 ERROR querymaster.QueryUnitAttempt (QueryUnitAttempt.java:transition(405)) - FROM 192.168.0.11 >> java.lang.NullPointerException
      	at org.apache.tajo.engine.eval.EvalTreeUtil$EvalReplaceVisitor.visitChild(EvalTreeUtil.java:60)
      	at org.apache.tajo.engine.eval.EvalTreeUtil$EvalReplaceVisitor.visitChild(EvalTreeUtil.java:44)
      	at org.apache.tajo.engine.eval.BasicEvalNodeVisitor.visitDefaultFunctionEval(BasicEvalNodeVisitor.java:154)
      	at org.apache.tajo.engine.eval.BasicEvalNodeVisitor.visitFuncCall(BasicEvalNodeVisitor.java:316)
      	at org.apache.tajo.engine.eval.BasicEvalNodeVisitor.visitChild(BasicEvalNodeVisitor.java:121)
      	at org.apache.tajo.engine.eval.EvalTreeUtil$EvalReplaceVisitor.visitChild(EvalTreeUtil.java:55)
      	at org.apache.tajo.engine.eval.EvalTreeUtil.replace(EvalTreeUtil.java:41)
      	at org.apache.tajo.engine.planner.physical.SeqScanExec.rewriteColumnPartitionedTableSchema(SeqScanExec.java:115)
      	at org.apache.tajo.engine.planner.physical.SeqScanExec.init(SeqScanExec.java:131)
      	at org.apache.tajo.engine.planner.physical.UnaryPhysicalExec.init(UnaryPhysicalExec.java:52)
      	at org.apache.tajo.engine.planner.physical.StoreTableExec.init(StoreTableExec.java:49)
      	at org.apache.tajo.worker.Task.run(Task.java:383)
      	at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:391)
      	at java.lang.Thread.run(Thread.java:744)
      
      
      1. TAJO-777_2.patch
        22 kB
        Hyunsik Choi
      2. TAJO-777.patch
        6 kB
        Hyoungjun Kim

        Activity

        Hide
        hjkim Hyoungjun Kim added a comment - - edited

        Created a review request against branch master in reviewboard
        https://reviews.apache.org/r/20513/

        Show
        hjkim Hyoungjun Kim added a comment - - edited Created a review request against branch master in reviewboard https://reviews.apache.org/r/20513/
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12640934/TAJO-777.patch
        against master revision f8ba4db.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 8 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-core.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/359//testReport/
        Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/359//findbugsResult
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/359//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12640934/TAJO-777.patch against master revision f8ba4db. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-core. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/359//testReport/ Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/359//findbugsResult Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/359//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment - - edited

        +1

        Thank you for the bug fix. It's a nice finding. In addition, I found more cases similar to the bug you found. The cases were in BetweenPredicateEval, CaseWhenEval, IfElseEval, CastEval, NotEval, and SignedEval. So, I've updated your patch to include more bug fixes for them and the unit tests to reproduce the cases.

        If you agree with this patch or another committer throws +1, I'll commit it. Thanks!

        Show
        hyunsik Hyunsik Choi added a comment - - edited +1 Thank you for the bug fix. It's a nice finding. In addition, I found more cases similar to the bug you found. The cases were in BetweenPredicateEval, CaseWhenEval, IfElseEval, CastEval, NotEval, and SignedEval. So, I've updated your patch to include more bug fixes for them and the unit tests to reproduce the cases. If you agree with this patch or another committer throws +1, I'll commit it. Thanks!
        Hide
        hjkim Hyoungjun Kim added a comment -

        +1 Hyunsik Thanks.

        Show
        hjkim Hyoungjun Kim added a comment - +1 Hyunsik Thanks.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-build #188 (See https://builds.apache.org/job/Tajo-master-build/188/)
        TAJO-777: Partition column in function parameter occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev e56a7a41fb6b27e093c5c49a2d86a8cabc203156)

        • tajo-core/src/test/resources/results/TestTablePartitions/case7.result
        • tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLQuotedTableName2.result
        • CHANGES.txt
        • tajo-core/src/test/resources/queries/TestTablePartitions/case8.sql
        • tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case8.result
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/CaseWhenEval.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/EvalTreeUtil.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/SignedEval.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case10.result
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/DDLBuilder.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/NotEval.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case10.sql
        • tajo-core/src/test/resources/results/TestTajoDump/testDump1.result
        • tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLForBaseTable.result
        • tajo-core/src/test/resources/queries/TestTablePartitions/case9.sql
        • tajo-core/src/test/java/org/apache/tajo/client/TestTajoDump.java
        • tajo-core/src/test/resources/org/apache/tajo/client/TestTajoDump.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/BetweenPredicateEval.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case9.result
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/BasicEvalNodeVisitor.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/CastEval.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #188 (See https://builds.apache.org/job/Tajo-master-build/188/ ) TAJO-777 : Partition column in function parameter occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev e56a7a41fb6b27e093c5c49a2d86a8cabc203156) tajo-core/src/test/resources/results/TestTablePartitions/case7.result tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLQuotedTableName2.result CHANGES.txt tajo-core/src/test/resources/queries/TestTablePartitions/case8.sql tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java tajo-core/src/test/resources/results/TestTablePartitions/case8.result tajo-core/src/main/java/org/apache/tajo/engine/eval/CaseWhenEval.java tajo-core/src/main/java/org/apache/tajo/engine/eval/EvalTreeUtil.java tajo-core/src/main/java/org/apache/tajo/engine/eval/SignedEval.java tajo-core/src/test/resources/results/TestTablePartitions/case10.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/DDLBuilder.java tajo-core/src/main/java/org/apache/tajo/engine/eval/NotEval.java tajo-core/src/test/resources/queries/TestTablePartitions/case10.sql tajo-core/src/test/resources/results/TestTajoDump/testDump1.result tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLForBaseTable.result tajo-core/src/test/resources/queries/TestTablePartitions/case9.sql tajo-core/src/test/java/org/apache/tajo/client/TestTajoDump.java tajo-core/src/test/resources/org/apache/tajo/client/TestTajoDump.java tajo-core/src/main/java/org/apache/tajo/engine/eval/BetweenPredicateEval.java tajo-core/src/test/resources/results/TestTablePartitions/case9.result tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java tajo-core/src/main/java/org/apache/tajo/engine/eval/BasicEvalNodeVisitor.java tajo-core/src/main/java/org/apache/tajo/engine/eval/CastEval.java tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #189 (See https://builds.apache.org/job/Tajo-master-build/189/)
        TAJO-777: Partition column in function parameter occurs NPE. (missed test files) (hyunsik: rev f1f36ec5f5ef78168dfe024c07c1ba122f2d78c6)

        • tajo-core/src/test/resources/queries/TestTablePartitions/case5.sql
        • tajo-core/src/test/resources/queries/TestTablePartitions/case6.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case5.result
        • tajo-core/src/test/resources/queries/TestTablePartitions/case4.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case4.result
        • tajo-core/src/test/resources/queries/TestTablePartitions/case7.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case6.result
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #189 (See https://builds.apache.org/job/Tajo-master-build/189/ ) TAJO-777 : Partition column in function parameter occurs NPE. (missed test files) (hyunsik: rev f1f36ec5f5ef78168dfe024c07c1ba122f2d78c6) tajo-core/src/test/resources/queries/TestTablePartitions/case5.sql tajo-core/src/test/resources/queries/TestTablePartitions/case6.sql tajo-core/src/test/resources/results/TestTablePartitions/case5.result tajo-core/src/test/resources/queries/TestTablePartitions/case4.sql tajo-core/src/test/resources/results/TestTablePartitions/case4.result tajo-core/src/test/resources/queries/TestTablePartitions/case7.sql tajo-core/src/test/resources/results/TestTablePartitions/case6.result
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed it to master and 0.8.0 branch.

        Show
        hyunsik Hyunsik Choi added a comment - committed it to master and 0.8.0 branch.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-0.8.0-build #81 (See https://builds.apache.org/job/Tajo-0.8.0-build/81/)
        TAJO-777: Partition column in function parameter occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev 8af6e3e6a35ce0a35f84902869d9a5386917aaa3)

        • tajo-core/src/main/java/org/apache/tajo/engine/eval/BasicEvalNodeVisitor.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case9.result
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/NotEval.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case4.result
        • tajo-core/src/test/resources/queries/TestTablePartitions/case7.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case7.result
        • tajo-core/src/test/resources/org/apache/tajo/client/TestTajoDump.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case10.sql
        • tajo-core/src/test/resources/results/TestTajoDump/testDump1.result
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/EvalTreeUtil.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case4.sql
        • tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case6.result
        • CHANGES.txt
        • tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLForBaseTable.result
        • tajo-core/src/test/java/org/apache/tajo/client/TestTajoDump.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/CaseWhenEval.java
        • tajo-core/src/test/resources/results/TestTablePartitions/case10.result
        • tajo-core/src/test/resources/queries/TestTablePartitions/case8.sql
        • tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLQuotedTableName2.result
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/CastEval.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case9.sql
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/BetweenPredicateEval.java
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/DDLBuilder.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case6.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case8.result
        • tajo-core/src/main/java/org/apache/tajo/engine/eval/SignedEval.java
        • tajo-core/src/test/resources/queries/TestTablePartitions/case5.sql
        • tajo-core/src/test/resources/results/TestTablePartitions/case5.result
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.8.0-build #81 (See https://builds.apache.org/job/Tajo-0.8.0-build/81/ ) TAJO-777 : Partition column in function parameter occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev 8af6e3e6a35ce0a35f84902869d9a5386917aaa3) tajo-core/src/main/java/org/apache/tajo/engine/eval/BasicEvalNodeVisitor.java tajo-core/src/test/resources/results/TestTablePartitions/case9.result tajo-core/src/main/java/org/apache/tajo/engine/eval/NotEval.java tajo-core/src/test/resources/results/TestTablePartitions/case4.result tajo-core/src/test/resources/queries/TestTablePartitions/case7.sql tajo-core/src/test/resources/results/TestTablePartitions/case7.result tajo-core/src/test/resources/org/apache/tajo/client/TestTajoDump.java tajo-core/src/test/resources/queries/TestTablePartitions/case10.sql tajo-core/src/test/resources/results/TestTajoDump/testDump1.result tajo-core/src/main/java/org/apache/tajo/engine/eval/EvalTreeUtil.java tajo-core/src/test/resources/queries/TestTablePartitions/case4.sql tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java tajo-core/src/test/resources/results/TestTablePartitions/case6.result CHANGES.txt tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLForBaseTable.result tajo-core/src/test/java/org/apache/tajo/client/TestTajoDump.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java tajo-core/src/main/java/org/apache/tajo/engine/eval/CaseWhenEval.java tajo-core/src/test/resources/results/TestTablePartitions/case10.result tajo-core/src/test/resources/queries/TestTablePartitions/case8.sql tajo-core/src/test/resources/results/testDDLBuilder/testBuildDDLQuotedTableName2.result tajo-core/src/main/java/org/apache/tajo/engine/eval/CastEval.java tajo-core/src/test/resources/queries/TestTablePartitions/case9.sql tajo-core/src/main/java/org/apache/tajo/engine/eval/BetweenPredicateEval.java tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/DDLBuilder.java tajo-core/src/test/resources/queries/TestTablePartitions/case6.sql tajo-core/src/test/resources/results/TestTablePartitions/case8.result tajo-core/src/main/java/org/apache/tajo/engine/eval/SignedEval.java tajo-core/src/test/resources/queries/TestTablePartitions/case5.sql tajo-core/src/test/resources/results/TestTablePartitions/case5.result

          People

          • Assignee:
            hjkim Hyoungjun Kim
            Reporter:
            hjkim Hyoungjun Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development