Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1339

International characters in column names not supported

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0, 0.7.0, 0.8.0
    • None
    • impl
    • None

    Description

      There is a particular use-case in which someone specifies a column name to be in International characters.

      inputdata = load '/user/viraj/inputdata.txt' using PigStorage() as (あいうえお);
      describe inputdata;
      dump inputdata;
      

      ======================================================================================================
      Pig Stack Trace
      ---------------
      ERROR 1000: Error during parsing. Lexical error at line 1, column 64. Encountered: "\u3042" (12354), after : ""

      org.apache.pig.impl.logicalLayer.parser.TokenMgrError: Lexical error at line 1, column 64. Encountered: "\u3042" (12354), after : ""

      at org.apache.pig.impl.logicalLayer.parser.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1791)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_scan_token(QueryParser.java:8959)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3R_51(QueryParser.java:7462)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3R_120(QueryParser.java:7769)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3R_106(QueryParser.java:7787)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3R_63(QueryParser.java:8609)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3R_32(QueryParser.java:8621)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_3_4(QueryParser.java:8354)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.jj_2_4(QueryParser.java:6903)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1249)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:911)
      at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:700)
      at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
      at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164)
      at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114)
      at org.apache.pig.PigServer.registerQuery(PigServer.java:425)
      at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737)
      at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324)
      at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
      at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
      at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
      at org.apache.pig.Main.main(Main.java:391)
      ======================================================================================================

      Thanks Viraj

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              viraj Viraj Bhat
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated: