Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-2080

ArrayIndexOutOfBoundsException when performing aggregation on an union block

    Details

      Description

      Here is the log.

      default> select sum(t.cnt) as cnt, l_orderkey, l_partkey from (
      select l_orderkey, l_partkey, CAST(COUNT(1) AS INT4) as cnt from lineitem_small group by l_orderkey, l_partkey 
      union all 
      select l_orderkey, l_partkey, l_linenumber from lineitem) as t 
      group by l_orderkey, l_partkey;
      
      2016-02-18 15:49:37,162 WARN: org.apache.tajo.catalog.statistics.StatisticsUtil (aggregateTableStat(136)) - 3
      java.lang.ArrayIndexOutOfBoundsException: 3
          at org.apache.tajo.catalog.statistics.StatisticsUtil.aggregateTableStat(StatisticsUtil.java:125)
          at org.apache.tajo.querymaster.Repartitioner.computeChildBlocksStats(Repartitioner.java:627)
          at org.apache.tajo.querymaster.Repartitioner.scheduleHashShuffledFetches(Repartitioner.java:899)
          at org.apache.tajo.querymaster.Repartitioner.scheduleFragmentsForNonLeafTasks(Repartitioner.java:611)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.schedule(Stage.java:1062)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.access$900(Stage.java:839)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer$1.run(Stage.java:865)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
      2016-02-18 15:49:37,162 WARN: org.apache.tajo.catalog.statistics.StatisticsUtil (aggregateTableStat(136)) - 4
      java.lang.ArrayIndexOutOfBoundsException: 4
          at org.apache.tajo.catalog.statistics.StatisticsUtil.aggregateTableStat(StatisticsUtil.java:125)
          at org.apache.tajo.querymaster.Repartitioner.computeChildBlocksStats(Repartitioner.java:627)
          at org.apache.tajo.querymaster.Repartitioner.scheduleHashShuffledFetches(Repartitioner.java:899)
          at org.apache.tajo.querymaster.Repartitioner.scheduleFragmentsForNonLeafTasks(Repartitioner.java:611)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.schedule(Stage.java:1062)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.access$900(Stage.java:839)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer$1.run(Stage.java:865)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
      2016-02-18 15:49:37,163 WARN: org.apache.tajo.catalog.statistics.StatisticsUtil (aggregateTableStat(136)) - 5
      java.lang.ArrayIndexOutOfBoundsException: 5
          at org.apache.tajo.catalog.statistics.StatisticsUtil.aggregateTableStat(StatisticsUtil.java:125)
          at org.apache.tajo.querymaster.Repartitioner.computeChildBlocksStats(Repartitioner.java:627)
          at org.apache.tajo.querymaster.Repartitioner.scheduleHashShuffledFetches(Repartitioner.java:899)
          at org.apache.tajo.querymaster.Repartitioner.scheduleFragmentsForNonLeafTasks(Repartitioner.java:611)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.schedule(Stage.java:1062)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.access$900(Stage.java:839)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer$1.run(Stage.java:865)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
      ...
      

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jihoonson opened a pull request:

        https://github.com/apache/tajo/pull/967

        TAJO-2080: ArrayIndexOutOfBoundsException when performing aggregation…

        … on an union block.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jihoonson/tajo-2 TAJO-2080

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/967.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #967


        commit f7b440e3046e560c50ee627797bd8d4f76b66c62
        Author: Jihoon Son <jihoonson@apache.org>
        Date: 2016-02-20T12:07:32Z

        TAJO-2080: ArrayIndexOutOfBoundsException when performing aggregation on an union block.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jihoonson opened a pull request: https://github.com/apache/tajo/pull/967 TAJO-2080 : ArrayIndexOutOfBoundsException when performing aggregation… … on an union block. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jihoonson/tajo-2 TAJO-2080 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/967.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #967 commit f7b440e3046e560c50ee627797bd8d4f76b66c62 Author: Jihoon Son <jihoonson@apache.org> Date: 2016-02-20T12:07:32Z TAJO-2080 : ArrayIndexOutOfBoundsException when performing aggregation on an union block.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/967#issuecomment-186968465

        +1 LGTM

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/967#issuecomment-186968465 +1 LGTM
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/967

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/967
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #685 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/685/)
        TAJO-2080: ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev 1138784fbd1e3e7b34ce616bd308250ef59ab859)

        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java
        • tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result
        • CHANGES
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #685 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/685/ ) TAJO-2080 : ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev 1138784fbd1e3e7b34ce616bd308250ef59ab859) tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result CHANGES
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #1093 (See https://builds.apache.org/job/Tajo-master-build/1093/)
        TAJO-2080: ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev 1138784fbd1e3e7b34ce616bd308250ef59ab859)

        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java
        • CHANGES
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #1093 (See https://builds.apache.org/job/Tajo-master-build/1093/ ) TAJO-2080 : ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev 1138784fbd1e3e7b34ce616bd308250ef59ab859) tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java CHANGES tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-0.11.2-build #174 (See https://builds.apache.org/job/Tajo-0.11.2-build/174/)
        TAJO-2080: ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev cd1b133a5dfb76090fd8f3a957492428044b3054)

        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
        • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java
        • tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result
        • tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java
        • CHANGES
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.11.2-build #174 (See https://builds.apache.org/job/Tajo-0.11.2-build/174/ ) TAJO-2080 : ArrayIndexOutOfBoundsException when performing aggregation on (jihoonson: rev cd1b133a5dfb76090fd8f3a957492428044b3054) tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java tajo-core-tests/src/test/resources/results/TestTableSubQuery/testGroupbyOnUnion.result tajo-core/src/main/java/org/apache/tajo/querymaster/Query.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java CHANGES

          People

          • Assignee:
            jihoonson Jihoon Son
            Reporter:
            jihoonson Jihoon Son
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development