Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8087

Evaluate to adapt the serde property values length as per the latest changes on HMS

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.7.0
    • Fix Version/s: None
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-1

      Description

      Hi,
      with the HIVE-12274 (C5.13) seems that the SERDE_PARAMS.PARAM_VALUE max length is increased in all the supported RDBMS to a bigger value while on our side we kept to 4k (depending by the RDBMS the datatype used increased the length between 32k and 64k).

      I saw that those lines are not changed in the latest releases:
      https://github.com/cloudera/Impala/blob/cdh5-2.7.0_5.10.2/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java#L50-L51
      https://github.com/cloudera/Impala/blob/cdh5-2.7.0_5.10.2/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java#L145-L155

      Not sure if can be done without breaking anything, however could you evaluate if it is possible to safely increase it to a bigger value? If that case could you get this Jira in your backlog?

      It will allow as example the creation of avro table with avro.schema.literal > 4k as in Hive instead to exit with:

       AnalysisException: Property value length must be <= 4000:
      

      Thank you,
      Adriano

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              adrenas Adriano
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: