Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1190

INSERT INTO to partition tables may cause NPE.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      Reproduction

      tajo> CREATE EXTERNAL TABLE distinct_tb (distinct_id text) USING CSV PARTITION BY COLUMN (CNT TEXT) LOCATION '/user/hyunsik/test';
      OK
      
      > insert overwrite into distinct_tb (distinct_id) select l_returnflag from lineitem where l_orderkey = 1000 limit 1;
      default> insert overwrite into distinct_tb (distinct_id) select l_returnflag from lineitem where l_orderkey = 1000 limit 1;
      Progress: 0%, response time: 0.202 sec
      Progress: 0%, response time: 0.203 sec
      Progress: 0%, response time: 0.607 sec
      Progress: 11%, response time: 1.409 sec
      Progress: 28%, response time: 2.414 sec
      ERROR: java.lang.NullPointerException
      java.io.IOException: java.lang.NullPointerException
      	at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:152)
      	at org.apache.tajo.worker.Task.run(Task.java:448)
      	at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:276)
      	at java.lang.Thread.run(Thread.java:744)
      Caused by: java.lang.NullPointerException
      	at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:142)
      	... 3 more
      

      Description
      This problem actually includes three bugs: 1) NULL of InputStats of SeqScan, 2) NULL of getCause().getMessage in RemoteCallException, and 3) some null check bug in TajoWorkerClientService.

      This patch fixes the all bugs that I found.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user hyunsik opened a pull request:

        https://github.com/apache/tajo/pull/250

        TAJO-1190: INSERT INTO to partition tables may cause NPE.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/hyunsik/tajo TAJO-1190

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/250.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #250


        commit 8822b71ce5dd4d96be32329e44196ee452b6f8fd
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2014-11-18T04:44:44Z

        TAJO-1190: INSERT INTO to partition tables may cause NPE.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/250 TAJO-1190 : INSERT INTO to partition tables may cause NPE. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1190 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/250.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #250 commit 8822b71ce5dd4d96be32329e44196ee452b6f8fd Author: Hyunsik Choi <hyunsik@apache.org> Date: 2014-11-18T04:44:44Z TAJO-1190 : INSERT INTO to partition tables may cause NPE.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/250#issuecomment-63754237

        +1

        It looks good to me and all unit test cases passed successfully.
        Push it.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/250#issuecomment-63754237 +1 It looks good to me and all unit test cases passed successfully. Push it.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/250#issuecomment-65177520

        Hi @blrunner,

        I cleaned up and I've fixed more bugs. This patch also includes TAJO-1219 (https://issues.apache.org/jira/browse/TAJO-1219). Could you review it again?

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/250#issuecomment-65177520 Hi @blrunner, I cleaned up and I've fixed more bugs. This patch also includes TAJO-1219 ( https://issues.apache.org/jira/browse/TAJO-1219 ). Could you review it again?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/250#issuecomment-65241039

        +1
        The fix looks straightforward. Travis-CI seem busy.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/250#issuecomment-65241039 +1 The fix looks straightforward. Travis-CI seem busy.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/250#issuecomment-65267554

        Thanks! I'll commit it shortly. In some cases, I may need the jenkins instead of TravisCI. Of course, I'll submit the patch through PR.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/250#issuecomment-65267554 Thanks! I'll commit it shortly. In some cases, I may need the jenkins instead of TravisCI. Of course, I'll submit the patch through PR.
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed. Thank you for the reviews.

        Show
        hyunsik Hyunsik Choi added a comment - committed. Thank you for the reviews.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/250

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/250
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #474 (See https://builds.apache.org/job/Tajo-master-build/474/)
        TAJO-1190: INSERT INTO to partition tables may cause NPE. (hyunsik: rev 1cdbe467e3dc25d7af59afc116ff9e8e6273a1ac)

        • CHANGES
        • tajo-rpc/src/main/java/org/apache/tajo/rpc/RemoteCallException.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions5.result
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions6.result
        • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #474 (See https://builds.apache.org/job/Tajo-master-build/474/ ) TAJO-1190 : INSERT INTO to partition tables may cause NPE. (hyunsik: rev 1cdbe467e3dc25d7af59afc116ff9e8e6273a1ac) CHANGES tajo-rpc/src/main/java/org/apache/tajo/rpc/RemoteCallException.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions5.result tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions6.result tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #115 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/115/)
        TAJO-1190: INSERT INTO to partition tables may cause NPE. (hyunsik: rev 1cdbe467e3dc25d7af59afc116ff9e8e6273a1ac)

        • tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions5.result
        • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
        • tajo-rpc/src/main/java/org/apache/tajo/rpc/RemoteCallException.java
        • CHANGES
        • tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions6.result
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #115 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/115/ ) TAJO-1190 : INSERT INTO to partition tables may cause NPE. (hyunsik: rev 1cdbe467e3dc25d7af59afc116ff9e8e6273a1ac) tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions5.result tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-rpc/src/main/java/org/apache/tajo/rpc/RemoteCallException.java CHANGES tajo-core/src/test/resources/results/TestTablePartitions/testColumnPartitionedTableWithSmallerExpressions6.result tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development