Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14085

Allow type widening primitive conversion on hive/parquet tables

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: File Formats
    • Labels:
      None

      Description

      There is a JIRA ticket on upstream that brought this usability improvement in Hive to support auto type widening for Parquet tables. See https://issues.apache.org/jira/browse/HIVE-12080
      This improvement is very useful for users who have schema evolution on their tables. For example, a Hive table with a "bigint" can read parquet files with "int32" and "int64" types.
      The patch only supports widening conversions from int->bigint and float->double. We should support more types to allow users read their changed parquet schema.
      Here's a list of widening conversions we should support:

      tinyint     ->  smallint,int,bigint,float,double
      smallint  -> int,bigint,float,double
      int          -> bigint,float,double
      bigint     -> float,double
      float       ->  double
      double   ->  --
      

        Attachments

        1. HIVE-14085.1.patch
          19 kB
          Vihang Karajgaonkar

          Activity

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar
              Reporter:
              vihangk1 Vihang Karajgaonkar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: