Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-452

Timstamp literal with fractional seconds results in java.lang.ArrayIndexOutOfBoundsException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Frational seconds with Timestamp literal induces java.lang.ArrayIndexOutOfBoundsException as followings:

      select TIMESTAMP '1970-01-17 10:09:37.5';
      
      java.lang.ArrayIndexOutOfBoundsException: 0
      	at org.apache.tajo.engine.parser.SQLAnalyzer.parseTime(SQLAnalyzer.java:1316)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitTimestamp_literal(SQLAnalyzer.java:1299)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitDatetime_literal(SQLAnalyzer.java:1290)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitGeneral_literal(SQLAnalyzer.java:1282)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitUnsigned_literal(SQLAnalyzer.java:1273)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitUnsigned_literal(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Unsigned_literalContext.accept(SQLParser.java:2048)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitUnsigned_value_specification(SQLAnalyzer.java:1265)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitUnsigned_value_specification(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Unsigned_value_specificationContext.accept(SQLParser.java:3858)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNonparenthesized_value_expression_primary(SQLAnalyzer.java:570)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNonparenthesized_value_expression_primary(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Nonparenthesized_value_expression_primaryContext.accept(SQLParser.java:3761)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitValue_expression_primary(SQLParserBaseVisitor.java:1047)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNumeric_primary(SQLAnalyzer.java:653)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitFactor(SQLAnalyzer.java:644)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitTerm(SQLAnalyzer.java:622)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNumeric_value_expression(SQLAnalyzer.java:601)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNumeric_value_expression(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Numeric_value_expressionContext.accept(SQLParser.java:5368)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitCommon_value_expression(SQLAnalyzer.java:473)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitCommon_value_expression(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Common_value_expressionContext.accept(SQLParser.java:5293)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitValue_expression(SQLParserBaseVisitor.java:1447)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitDerived_column(SQLAnalyzer.java:826)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitSelect_sublist(SQLAnalyzer.java:258)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitSelect_list(SQLAnalyzer.java:235)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitQuery_specification(SQLAnalyzer.java:202)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitQuery_specification(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Query_specificationContext.accept(SQLParser.java:9571)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitSimple_table(SQLParserBaseVisitor.java:415)
      	at org.apache.tajo.engine.parser.SQLParser$Simple_tableContext.accept(SQLParser.java:9340)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitNon_join_query_primary(SQLParserBaseVisitor.java:103)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNon_join_query_term(SQLAnalyzer.java:130)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNon_join_query_expression(SQLAnalyzer.java:85)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitNon_join_query_expression(SQLAnalyzer.java:47)
      	at org.apache.tajo.engine.parser.SQLParser$Non_join_query_expressionContext.accept(SQLParser.java:8928)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitQuery_expression_body(SQLParserBaseVisitor.java:887)
      	at org.apache.tajo.engine.parser.SQLParser$Query_expression_bodyContext.accept(SQLParser.java:8848)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitQuery_expression(SQLParserBaseVisitor.java:31)
      	at org.apache.tajo.engine.parser.SQLParser$Query_expressionContext.accept(SQLParser.java:8802)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitData_statement(SQLParserBaseVisitor.java:727)
      	at org.apache.tajo.engine.parser.SQLParser$Data_statementContext.accept(SQLParser.java:369)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:68)
      	at org.apache.tajo.engine.parser.SQLParserBaseVisitor.visitStatement(SQLParserBaseVisitor.java:1023)
      	at org.apache.tajo.engine.parser.SQLParser$StatementContext.accept(SQLParser.java:300)
      	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:44)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.visitSql(SQLAnalyzer.java:79)
      	at org.apache.tajo.engine.parser.SQLAnalyzer.parse(SQLAnalyzer.java:70)
      	at org.apache.tajo.engine.eval.ExprTestBase.getRawTargets(ExprTestBase.java:78)
      	at org.apache.tajo.engine.eval.ExprTestBase.testEval(ExprTestBase.java:123)
      	at org.apache.tajo.engine.eval.ExprTestBase.testSimpleEval(ExprTestBase.java:91)
      	at org.apache.tajo.engine.function.TestDateTimeFunctions.testExtract(TestDateTimeFunctions.java:57)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
      
      1. TAJO-452.patch
        5 kB
        Keuntae Park

        Activity

        Hide
        sirpkt Keuntae Park added a comment - - edited

        I've uploaded the patch for the issue

        • It fixes the bug in parseTime() method of SQLAnalyzer class: the input of String.split() is interpreted as regular expression, so it should be
          String [] secondsParts = parts[2].split("\\.");
        • It enhances TimestampDatum class to be able to show fractional seconds. However, accuracy is limit to three decimal places because of the limitation of Joda-Time that Tajo Timestamp relies on.
        • deserialize(), serialize() methods of DatumAdapter class are also changed to preserve the fractional seconds.
        • It also adds testToTimestamp() in TestSQLDateTimeTypes.java

        Please review this patch.

        Show
        sirpkt Keuntae Park added a comment - - edited I've uploaded the patch for the issue It fixes the bug in parseTime() method of SQLAnalyzer class: the input of String.split() is interpreted as regular expression, so it should be String [] secondsParts = parts[2].split( "\\." ); It enhances TimestampDatum class to be able to show fractional seconds. However, accuracy is limit to three decimal places because of the limitation of Joda-Time that Tajo Timestamp relies on. deserialize(), serialize() methods of DatumAdapter class are also changed to preserve the fractional seconds. It also adds testToTimestamp() in TestSQLDateTimeTypes.java Please review this patch.
        Hide
        charsyam DaeMyung Kang added a comment -

        Keuntae Park Great Catch.

        You're right. ant the patch looks good to me.

        and "mvn clean install" is OK.

        Show
        charsyam DaeMyung Kang added a comment - Keuntae Park Great Catch. You're right. ant the patch looks good to me. and "mvn clean install" is OK.
        Hide
        jihoonson Jihoon Son added a comment -

        +1.
        I'll commit if there aren't any objections for a while.

        Show
        jihoonson Jihoon Son added a comment - +1. I'll commit if there aren't any objections for a while.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I'm getting some unit test failure.

        Results :
        
        Tests in error: 
          testTimestamp(org.apache.tajo.engine.eval.TestSQLDateTimeTypes): Value 500 for secondOfMinute must be in the range [0,59]
        
        Tests run: 474, Failures: 0, Errors: 1, Skipped: 0
        
        Show
        hyunsik Hyunsik Choi added a comment - I'm getting some unit test failure. Results : Tests in error: testTimestamp(org.apache.tajo.engine.eval.TestSQLDateTimeTypes): Value 500 for secondOfMinute must be in the range [0,59] Tests run: 474, Failures: 0, Errors: 1, Skipped: 0
        Hide
        charsyam DaeMyung Kang added a comment -

        Hyunsik Choi I'm sorry. this is the reason of this unittest failure

        https://issues.apache.org/jira/browse/TAJO-454

        Show
        charsyam DaeMyung Kang added a comment - Hyunsik Choi I'm sorry. this is the reason of this unittest failure https://issues.apache.org/jira/browse/TAJO-454
        Hide
        hyunsik Hyunsik Choi added a comment -

        It's Ok. Could you submit the bug fix?

        Show
        hyunsik Hyunsik Choi added a comment - It's Ok. Could you submit the bug fix?
        Hide
        sirpkt Keuntae Park added a comment - - edited

        So, as TAJO-454 is committed, is everything OK now?

        'mvn clean install' passed successfully.

        Show
        sirpkt Keuntae Park added a comment - - edited So, as TAJO-454 is committed, is everything OK now? 'mvn clean install' passed successfully.
        Hide
        hyunsik Hyunsik Choi added a comment -

        +1

        Ship it!

        Show
        hyunsik Hyunsik Choi added a comment - +1 Ship it!
        Hide
        charsyam DaeMyung Kang added a comment -

        +1
        It's ok. and "mvn clean install" is OK.
        Sorry Keuntae Park

        Show
        charsyam DaeMyung Kang added a comment - +1 It's ok. and "mvn clean install" is OK. Sorry Keuntae Park
        Hide
        sirpkt Keuntae Park added a comment -

        DaeMyung Kang No, problem Thank you for the review.

        Show
        sirpkt Keuntae Park added a comment - DaeMyung Kang No, problem Thank you for the review.
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed the bug fix to master branch. Thank you guys for the patch and reviews.

        Show
        hyunsik Hyunsik Choi added a comment - committed the bug fix to master branch. Thank you guys for the patch and reviews.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #646 (See https://builds.apache.org/job/Tajo-trunk-postcommit/646/)
        TAJO-452: Timstamp literal with fractional seconds results in java.lang.ArrayIndexOutOfBoundsException. (Keuntae Park via hyunsik). (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=319a37725feb5fe4fa04280da4fd11fd24406960)

        • CHANGES.txt
        • tajo-common/src/main/java/org/apache/tajo/json/DatumAdapter.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java
        • tajo-algebra/src/main/java/org/apache/tajo/algebra/TimeValue.java
        • tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/eval/TestSQLDateTimeTypes.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #646 (See https://builds.apache.org/job/Tajo-trunk-postcommit/646/ ) TAJO-452 : Timstamp literal with fractional seconds results in java.lang.ArrayIndexOutOfBoundsException. (Keuntae Park via hyunsik). (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=319a37725feb5fe4fa04280da4fd11fd24406960 ) CHANGES.txt tajo-common/src/main/java/org/apache/tajo/json/DatumAdapter.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java tajo-algebra/src/main/java/org/apache/tajo/algebra/TimeValue.java tajo-common/src/main/java/org/apache/tajo/datum/TimestampDatum.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/eval/TestSQLDateTimeTypes.java

          People

          • Assignee:
            sirpkt Keuntae Park
            Reporter:
            sirpkt Keuntae Park
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development