Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1777

JsonLineDeserializer returns invalid unicode text, if contains control character

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.10.1
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: Storage
    • Labels:
      None

      Description

      Currently, Json store uses json-smart. If input value is byte array of unicode and contains control character, json parser can be decoded to the invalid characters

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/696

        TAJO-1777: JsonLineDeserializer returns invalid unicode text, if cont…

        …ains control character

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-1777

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/696.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #696


        commit 31ba465ae8cacea201f1259b2f2ba3febacad98e
        Author: Jinho Kim <jhkim@apache.org>
        Date: 2015-08-17T05:36:00Z

        TAJO-1777: JsonLineDeserializer returns invalid unicode text, if contains control character


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/696 TAJO-1777 : JsonLineDeserializer returns invalid unicode text, if cont… …ains control character You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1777 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/696.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #696 commit 31ba465ae8cacea201f1259b2f2ba3febacad98e Author: Jinho Kim <jhkim@apache.org> Date: 2015-08-17T05:36:00Z TAJO-1777 : JsonLineDeserializer returns invalid unicode text, if contains control character
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/696#issuecomment-131907641

        +1 LGTM

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/696#issuecomment-131907641 +1 LGTM
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/696

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/696
        Hide
        jhkim Jinho Kim added a comment -

        committed it
        Thanks for your review.

        Show
        jhkim Jinho Kim added a comment - committed it Thanks for your review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #444 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/444/)
        TAJO-1777: JsonLineDeserializer returns invalid unicode text, if contains control character (jhkim: rev 57be230e921ca93029181a9983684d2fa80f8f06)

        • CHANGES
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java
        • tajo-storage/tajo-storage-hdfs/src/test/resources/dataset/TestJsonSerDe/testUnicodeWithControlChar.json
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #444 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/444/ ) TAJO-1777 : JsonLineDeserializer returns invalid unicode text, if contains control character (jhkim: rev 57be230e921ca93029181a9983684d2fa80f8f06) CHANGES tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java tajo-storage/tajo-storage-hdfs/src/test/resources/dataset/TestJsonSerDe/testUnicodeWithControlChar.json
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-build #805 (See https://builds.apache.org/job/Tajo-master-build/805/)
        TAJO-1777: JsonLineDeserializer returns invalid unicode text, if contains control character (jhkim: rev 57be230e921ca93029181a9983684d2fa80f8f06)

        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java
        • tajo-storage/tajo-storage-hdfs/src/test/resources/dataset/TestJsonSerDe/testUnicodeWithControlChar.json
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java
        • CHANGES
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-build #805 (See https://builds.apache.org/job/Tajo-master-build/805/ ) TAJO-1777 : JsonLineDeserializer returns invalid unicode text, if contains control character (jhkim: rev 57be230e921ca93029181a9983684d2fa80f8f06) tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java tajo-storage/tajo-storage-hdfs/src/test/resources/dataset/TestJsonSerDe/testUnicodeWithControlChar.json tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java CHANGES

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development