Uploaded image for project: 'HCatalog'
  1. HCatalog
  2. HCATALOG-436

JSON SerDe column misnaming on CTAS

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5, 0.4.1
    • Component/s: None
    • Labels:

      Description

      Given an origin table as follows:


      hive -e 'describe extended ttf'
      OK
      sterm string
      count bigint

      Detailed Table Information Table(tableName:ttf, dbName:default, owner:hive, createTime:1339518715, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:sterm, type:string, comment:null), FieldSchema(name:count, type:bigint, comment:null)], location:hdfs://localhost:54310/user/hive/warehouse/ttf, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:

      {serialization.format=1}

      ), bucketCols:[], sortCols:[], parameters:{}), partitionKeys:[], parameters:

      {numPartitions=0, numFiles=1, transient_lastDdlTime=1339518715, totalSize=2155, numRows=0, rawDataSize=0}

      , viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)

      On doing a CTAS, such as:


      hive -e "create table ttf_json row format serde 'org.apache.hcatalog.data.JsonSerDe' as select * from ttf;"

      We get a resultant table ttf_json with schema similar to ttf, but on looking at the data present in the json file itself, we'd notice data like this:

      {"_col0":"S8.66045288732867","_col1":103} {"_col0":"S8.66322678828148","_col1":95}

      This will then result in this table not being readable.

      This is behaviour similar to the one fixed in HCATALOG-275, but we've obviously not fixed all the possibilities of that problem.

        Attachments

        1. HCATALOG-436.patch
          5 kB
          Sushanth Sowmyan

          Issue Links

            Activity

              People

              • Assignee:
                sushanth Sushanth Sowmyan
                Reporter:
                sushanth Sushanth Sowmyan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: