Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Pending Closed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0, 0.10.1
    • Component/s: None
    • Labels:
      None

      Description

      Even though table's column have long char data, 'Char Datum' only returns 1byte.

      create table test1 (col1 char(5));
      insert into test1 select 'a';
      insert into test1 select 'abc';
      insert into test1 select 'abcde';
      select * from test1;
      col1
      -------------------------------
      a
      a
      a
      (3 rows, 0.151 sec, 6 B selected)

        Activity

        Hide
        dongjoon Dongjoon Hyun added a comment -

        Hi, Seungun Choe, this issue looks important for Tajo correctness!

        Show
        dongjoon Dongjoon Hyun added a comment - Hi, Seungun Choe , this issue looks important for Tajo correctness!
        Hide
        charsyam DaeMyung Kang added a comment -

        Seungun Choe Do you solve this issue?
        I found the reason why this result is weird.
        so final result like below. so If you allow, I will make a PR. Thank you.
        (I'm sorry to touch your issue.)

        default> create table test1 (col1 char(5));
        OK
        default> insert into test1 select 'a';
        (1 rows, 0.129 sec, 6 B inserted)
        default> insert into test1 select 'abc';
        (1 rows, 0.069 sec, 12 B inserted)
        default> insert into test1 select 'abcde';
        (1 rows, 0.081 sec, 18 B inserted)
        default> select * from test1;
        col1
        -------------------------------
        a
        abc
        abcde
        (3 rows, 0.043 sec, 18 B selected)

        Show
        charsyam DaeMyung Kang added a comment - Seungun Choe Do you solve this issue? I found the reason why this result is weird. so final result like below. so If you allow, I will make a PR. Thank you. (I'm sorry to touch your issue.) default> create table test1 (col1 char(5)); OK default> insert into test1 select 'a'; (1 rows, 0.129 sec, 6 B inserted) default> insert into test1 select 'abc'; (1 rows, 0.069 sec, 12 B inserted) default> insert into test1 select 'abcde'; (1 rows, 0.081 sec, 18 B inserted) default> select * from test1; col1 ------------------------------- a abc abcde (3 rows, 0.043 sec, 18 B selected)
        Hide
        seungunchoe Seungun Choe added a comment -

        DaeMyung Kang.
        Today, I'll solve this issue.
        But, if you solved this issue, attach patch file.

        Show
        seungunchoe Seungun Choe added a comment - DaeMyung Kang. Today, I'll solve this issue. But, if you solved this issue, attach patch file.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user charsyam opened a pull request:

        https://github.com/apache/tajo/pull/503

        TAJO-1485 Datum 'Char' returned only 1byte.

        It is because of Tajo-Client's encoding and decoding problem.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/charsyam/tajo feature/TAJO-1485

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/503.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #503


        commit b1e6aecfc88e38ffebd92881e1a932a56fb66b4c
        Author: clark.kang <clark.kang@kakao.com>
        Date: 2015-04-03T04:58:38Z

        TAJO-1485


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user charsyam opened a pull request: https://github.com/apache/tajo/pull/503 TAJO-1485 Datum 'Char' returned only 1byte. It is because of Tajo-Client's encoding and decoding problem. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charsyam/tajo feature/ TAJO-1485 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/503.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #503 commit b1e6aecfc88e38ffebd92881e1a932a56fb66b4c Author: clark.kang <clark.kang@kakao.com> Date: 2015-04-03T04:58:38Z TAJO-1485
        Hide
        seungunchoe Seungun Choe added a comment -

        DaeMyung Kang, assign this issue~

        Show
        seungunchoe Seungun Choe added a comment - DaeMyung Kang, assign this issue~
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/503#discussion_r27772807

        — Diff: tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java —
        @@ -197,7 +198,11 @@ private RowStoreEncoder(Schema schema) {
        case NULL_TYPE: nullFlags.set; break;
        case BOOLEAN: bb.put(tuple.get.asByte()); break;
        case BIT: bb.put(tuple.get.asByte()); break;

        • case CHAR: bb.put(tuple.get.asByte()); break;
          + case CHAR:
          + System.out.println("T1: " + tuple.get.size() + " " + tuple.get.asChars());
            • End diff –

        It seems to be a debug code. Could you delete this?

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on a diff in the pull request: https://github.com/apache/tajo/pull/503#discussion_r27772807 — Diff: tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java — @@ -197,7 +198,11 @@ private RowStoreEncoder(Schema schema) { case NULL_TYPE: nullFlags.set ; break; case BOOLEAN: bb.put(tuple.get .asByte()); break; case BIT: bb.put(tuple.get .asByte()); break; case CHAR: bb.put(tuple.get .asByte()); break; + case CHAR: + System.out.println("T1: " + tuple.get .size() + " " + tuple.get .asChars()); End diff – It seems to be a debug code. Could you delete this?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-89673377

        Could you add some unit test cases to TestTupleUtil?

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-89673377 Could you add some unit test cases to TestTupleUtil?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-89738589

        @blrunner Thank you for your review.
        I added unittest in TestTupleUtil and remove System.out debug code.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-89738589 @blrunner Thank you for your review. I added unittest in TestTupleUtil and remove System.out debug code.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-89770589

        @charsyam

        I found the updated patch and the unit test ran successfully. But I would like more better if you add one more unit test case to TestInsertQuery as follows.
        ```
        create table test1 (col1 char(5));
        insert into test1 select 'a';
        insert into test1 select 'abc';
        insert into test1 select 'abcde';
        select * from test1;
        col1
        -------------------------------
        a
        abc
        abcde
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-89770589 @charsyam I found the updated patch and the unit test ran successfully. But I would like more better if you add one more unit test case to TestInsertQuery as follows. ``` create table test1 (col1 char(5)); insert into test1 select 'a'; insert into test1 select 'abc'; insert into test1 select 'abcde'; select * from test1; col1 ------------------------------- a abc abcde ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-89777770

        @blrunner Thank you.
        I'll add unittest asap.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-89777770 @blrunner Thank you. I'll add unittest asap.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-92057398

        @blrunner I updated testcases and code

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-92057398 @blrunner I updated testcases and code
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-94344503

        Would you trigger Travis or Jenkins again after rebasing?

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-94344503 Would you trigger Travis or Jenkins again after rebasing?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-94358116

        @blrunner I rebased it, and it passed

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-94358116 @blrunner I rebased it, and it passed
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-99312899

        Thanks @charsyam

        I'll review this today.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-99312899 Thanks @charsyam I'll review this today.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-99941703

        Sorry for my late review.
        I failed to merge this as follows.
        ```
        Auto-merging tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
        CONFLICT (content): Merge conflict in tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
        Automatic merge failed; fix conflicts and then commit the result.
        ```
        Could you rebase this again?

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-99941703 Sorry for my late review. I failed to merge this as follows. ``` Auto-merging tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java CONFLICT (content): Merge conflict in tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java Automatic merge failed; fix conflicts and then commit the result. ``` Could you rebase this again?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-100139921

        Hi @charsyam, truly sorry for late review.
        Your patch looks good, but misses the default length of the char type. As you know, the length can be ignored when defining a schema, and it is handled as the char of the 1 byte length.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-100139921 Hi @charsyam, truly sorry for late review. Your patch looks good, but misses the default length of the char type. As you know, the length can be ignored when defining a schema, and it is handled as the char of the 1 byte length.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-100148985

        @jihoonson Thanks.
        but, I think it is already handled by
        https://github.com/apache/tajo/pull/531
        Is there any mistakes I missed?

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-100148985 @jihoonson Thanks. but, I think it is already handled by https://github.com/apache/tajo/pull/531 Is there any mistakes I missed?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-100163043

        It seems that some cases are missed.
        Here is a failure example.
        ```
        default> create table test1 (col1 char);
        OK
        default> insert into test1 select 'a';
        (1 rows, 0.23 sec, 319 B inserted)
        default> select * from test1;
        col1
        -------------------------------
        ERROR: java.lang.ArrayIndexOutOfBoundsException
        java.sql.SQLException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:786)
        at org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter.printResult(DefaultTajoCliOutputFormatter.java:105)
        at org.apache.tajo.cli.tsql.TajoCli.localQueryCompleted(TajoCli.java:558)
        at org.apache.tajo.cli.tsql.TajoCli.executeQuery(TajoCli.java:532)
        at org.apache.tajo.cli.tsql.TajoCli.executeParsedResults(TajoCli.java:447)
        at org.apache.tajo.cli.tsql.TajoCli.runShell(TajoCli.java:419)
        at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:692)
        Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:87)
        at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:780)
        ... 6 more
        Caused by: com.google.protobuf.ServiceException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.client.QueryClientImpl.fetchNextQueryResult(QueryClientImpl.java:370)
        at org.apache.tajo.client.TajoClientImpl.fetchNextQueryResult(TajoClientImpl.java:134)
        at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:69)
        ... 7 more
        ```
        This is the master log.
        ```
        2015-05-08 18:02:04,216 ERROR org.apache.tajo.master.TajoMasterClientService:
        java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at org.apache.tajo.storage.RowStoreUtil$RowStoreEncoder.toBytes(RowStoreUtil.java:203)
        at org.apache.tajo.master.exec.NonForwardQueryResultFileScanner.getNextRows(NonForwardQueryResultFileScanner.java:181)
        at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getQueryResultData(TajoMasterClientService.java:551)
        at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
        at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.channelRead0(BlockingRpcServer.java:100)
        at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.channelRead0(BlockingRpcServer.java:61)
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:182)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at org.apache.tajo.rpc.MonitorServerHandler.channelRead(MonitorServerHandler.java:70)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at java.lang.Thread.run(Thread.java:745)
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-100163043 It seems that some cases are missed. Here is a failure example. ``` default> create table test1 (col1 char); OK default> insert into test1 select 'a'; (1 rows, 0.23 sec, 319 B inserted) default> select * from test1; col1 ------------------------------- ERROR: java.lang.ArrayIndexOutOfBoundsException java.sql.SQLException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:786) at org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter.printResult(DefaultTajoCliOutputFormatter.java:105) at org.apache.tajo.cli.tsql.TajoCli.localQueryCompleted(TajoCli.java:558) at org.apache.tajo.cli.tsql.TajoCli.executeQuery(TajoCli.java:532) at org.apache.tajo.cli.tsql.TajoCli.executeParsedResults(TajoCli.java:447) at org.apache.tajo.cli.tsql.TajoCli.runShell(TajoCli.java:419) at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:692) Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:87) at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:780) ... 6 more Caused by: com.google.protobuf.ServiceException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.client.QueryClientImpl.fetchNextQueryResult(QueryClientImpl.java:370) at org.apache.tajo.client.TajoClientImpl.fetchNextQueryResult(TajoClientImpl.java:134) at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:69) ... 7 more ``` This is the master log. ``` 2015-05-08 18:02:04,216 ERROR org.apache.tajo.master.TajoMasterClientService: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.tajo.storage.RowStoreUtil$RowStoreEncoder.toBytes(RowStoreUtil.java:203) at org.apache.tajo.master.exec.NonForwardQueryResultFileScanner.getNextRows(NonForwardQueryResultFileScanner.java:181) at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getQueryResultData(TajoMasterClientService.java:551) at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549) at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.channelRead0(BlockingRpcServer.java:100) at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.channelRead0(BlockingRpcServer.java:61) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:182) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at org.apache.tajo.rpc.MonitorServerHandler.channelRead(MonitorServerHandler.java:70) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-101077701

        @jihoonson could you retry travis ci? I think it is broken with other issue not this

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-101077701 @jihoonson could you retry travis ci? I think it is broken with other issue not this
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-101492752

        Thanks @charsyam. The above bug looks fixed.
        Additionally, I have three comments as follows.

        • Please follow our convention for indent.
        • TestInsertQuery is a class to test Tajo's insert function rather than select result. I think that TestStorages looks a good place to test the character type for several types of storage.
        • Please consider the following case. I think that ```ValueTooLongForTypeCharactersException``` should be thrown instead of ```java.lang.ArrayIndexOutOfBoundsException```.

        ```
        default> \dfs -cat hdfs://localhost:7020/test
        cat: `hdfs://localhost:7020/test': Is a directory
        default> \dfs -cat hdfs://localhost:7020/test/test.csv
        1|a|1
        2|bb|2
        3|ccc|3
        4|dddd|4
        5|eeeee|5

        default> create external table test ( id int, name char(3), score int ) using text with ('text.delimiter'='|') location 'hdfs://localhost:7020/test';
        OK

        default> \d test

        table name: default.test
        table path: hdfs://localhost:7020/test
        store type: text
        number of rows: unknown
        volume: 40 B
        Options:
        'text.delimiter'='|'

        schema:
        id INT4
        name CHAR(3)
        score INT4

        default> select * from test;
        id, name, score
        -------------------------------
        ERROR: java.lang.ArrayIndexOutOfBoundsException
        java.sql.SQLException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:786)
        at org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter.printResult(DefaultTajoCliOutputFormatter.java:105)
        at org.apache.tajo.cli.tsql.TajoCli.localQueryCompleted(TajoCli.java:558)
        at org.apache.tajo.cli.tsql.TajoCli.executeQuery(TajoCli.java:532)
        at org.apache.tajo.cli.tsql.TajoCli.executeParsedResults(TajoCli.java:447)
        at org.apache.tajo.cli.tsql.TajoCli.runShell(TajoCli.java:419)
        at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:692)
        Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:87)
        at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:780)
        ... 6 more
        Caused by: com.google.protobuf.ServiceException: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.tajo.client.QueryClientImpl.fetchNextQueryResult(QueryClientImpl.java:370)
        at org.apache.tajo.client.TajoClientImpl.fetchNextQueryResult(TajoClientImpl.java:134)
        at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:69)
        ... 7 more
        ```

        Comments from the [review on Reviewable.io](https://reviewable.io:443/reviews/apache/tajo/503)
        <!-- Sent from Reviewable.io -->

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-101492752 Thanks @charsyam. The above bug looks fixed. Additionally, I have three comments as follows. Please follow our convention for indent. TestInsertQuery is a class to test Tajo's insert function rather than select result. I think that TestStorages looks a good place to test the character type for several types of storage. Please consider the following case. I think that ```ValueTooLongForTypeCharactersException``` should be thrown instead of ```java.lang.ArrayIndexOutOfBoundsException```. ``` default> \dfs -cat hdfs://localhost:7020/test cat: `hdfs://localhost:7020/test': Is a directory default> \dfs -cat hdfs://localhost:7020/test/test.csv 1|a|1 2|bb|2 3|ccc|3 4|dddd|4 5|eeeee|5 default> create external table test ( id int, name char(3), score int ) using text with ('text.delimiter'='|') location 'hdfs://localhost:7020/test'; OK default> \d test table name: default.test table path: hdfs://localhost:7020/test store type: text number of rows: unknown volume: 40 B Options: 'text.delimiter'='|' schema: id INT4 name CHAR(3) score INT4 default> select * from test; id, name, score ------------------------------- ERROR: java.lang.ArrayIndexOutOfBoundsException java.sql.SQLException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:786) at org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter.printResult(DefaultTajoCliOutputFormatter.java:105) at org.apache.tajo.cli.tsql.TajoCli.localQueryCompleted(TajoCli.java:558) at org.apache.tajo.cli.tsql.TajoCli.executeQuery(TajoCli.java:532) at org.apache.tajo.cli.tsql.TajoCli.executeParsedResults(TajoCli.java:447) at org.apache.tajo.cli.tsql.TajoCli.runShell(TajoCli.java:419) at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:692) Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:87) at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:780) ... 6 more Caused by: com.google.protobuf.ServiceException: java.lang.ArrayIndexOutOfBoundsException at org.apache.tajo.client.QueryClientImpl.fetchNextQueryResult(QueryClientImpl.java:370) at org.apache.tajo.client.TajoClientImpl.fetchNextQueryResult(TajoClientImpl.java:134) at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:69) ... 7 more ``` — Comments from the [review on Reviewable.io] ( https://reviewable.io:443/reviews/apache/tajo/503 ) <!-- Sent from Reviewable.io -->
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-101736590

        @jihoonson could you review new patch? I applied your review
        I moved storage test into TestStorages.
        but remained 2 tests into TestInsertQuery
        and fix indent. Thank you.

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-101736590 @jihoonson could you review new patch? I applied your review I moved storage test into TestStorages. but remained 2 tests into TestInsertQuery and fix indent. Thank you.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-101951303

        +1 LGTM!
        Thanks for your contribution.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-101951303 +1 LGTM! Thanks for your contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-build #705 (See https://builds.apache.org/job/Tajo-master-build/705/)
        TAJO-1485: Datum 'Char' returned only 1byte. (jihoonson: rev d3ca4bc1e59539896ddcb3b1075433a79f713114)

        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
        • tajo-common/src/main/java/org/apache/tajo/exception/ValueTooLongForTypeCharactersException.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/function/stream/TextFieldSerializerDeserializer.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/test1_nolength_ddl.sql
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TextSerializerDeserializer.java
        • tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-core/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/test1_ddl.sql
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeChar.sql
        • tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeCharWithNoLength.sql
        • CHANGES
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/BinarySerializerDeserializer.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #705 (See https://builds.apache.org/job/Tajo-master-build/705/ ) TAJO-1485 : Datum 'Char' returned only 1byte. (jihoonson: rev d3ca4bc1e59539896ddcb3b1075433a79f713114) tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-common/src/main/java/org/apache/tajo/exception/ValueTooLongForTypeCharactersException.java tajo-plan/src/main/java/org/apache/tajo/plan/function/stream/TextFieldSerializerDeserializer.java tajo-core/src/test/resources/queries/TestInsertQuery/test1_nolength_ddl.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TextSerializerDeserializer.java tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-core/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java tajo-core/src/test/resources/queries/TestInsertQuery/test1_ddl.sql tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeChar.sql tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeCharWithNoLength.sql CHANGES tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/BinarySerializerDeserializer.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #345 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/345/)
        TAJO-1485: Datum 'Char' returned only 1byte. (jihoonson: rev d3ca4bc1e59539896ddcb3b1075433a79f713114)

        • tajo-common/src/main/java/org/apache/tajo/exception/ValueTooLongForTypeCharactersException.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/test1_ddl.sql
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/test1_nolength_ddl.sql
        • CHANGES
        • tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeCharWithNoLength.sql
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
        • tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeChar.sql
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java
        • tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/function/stream/TextFieldSerializerDeserializer.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TextSerializerDeserializer.java
        • tajo-core/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/BinarySerializerDeserializer.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #345 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/345/ ) TAJO-1485 : Datum 'Char' returned only 1byte. (jihoonson: rev d3ca4bc1e59539896ddcb3b1075433a79f713114) tajo-common/src/main/java/org/apache/tajo/exception/ValueTooLongForTypeCharactersException.java tajo-core/src/test/resources/queries/TestInsertQuery/test1_ddl.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-core/src/test/resources/queries/TestInsertQuery/test1_nolength_ddl.sql CHANGES tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeCharWithNoLength.sql tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-core/src/test/resources/queries/TestInsertQuery/testInsertIntoSelectWithFixedSizeChar.sql tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoWriteSupport.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestInsertQuery.java tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/function/stream/TextFieldSerializerDeserializer.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextFieldSerializerDeserializer.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TextSerializerDeserializer.java tajo-core/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/BinarySerializerDeserializer.java
        Hide
        jihoonson Jihoon Son added a comment -

        Committed master and 0.10.1 branches.

        Show
        jihoonson Jihoon Son added a comment - Committed master and 0.10.1 branches.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/503#issuecomment-101954945

        I forgot closing this pr.
        Would you please close? Thanks.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/503#issuecomment-101954945 I forgot closing this pr. Would you please close? Thanks.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user charsyam closed the pull request at:

        https://github.com/apache/tajo/pull/503

        Show
        githubbot ASF GitHub Bot added a comment - Github user charsyam closed the pull request at: https://github.com/apache/tajo/pull/503

          People

          • Assignee:
            charsyam DaeMyung Kang
            Reporter:
            seungunchoe Seungun Choe
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development