[IMPALA-4282] Allow Impala to create columns whose type has more than 4000 characters

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
• Fix Version/s: Impala 2.9.0
• Component/s: Frontend
• Labels: newbie usability
• Target Version: Impala 2.9.0
• Assignee: Alexander Behm
• Reporter: Venkat Sambath
• Created: 12/Oct/16 09:02
• Updated: 17/Feb/17 06:08
• Resolved: 17/Feb/17 06:08

Description

When creating table with complex datatype having more than 4000 characters in hive we received the following error

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Add req_V2` (`CD_ID`,`COMMENT`,`COLUMN_NAME`,`TYPE_NAME`,`INTEGER_IDX`) VALUES (?,?,?,?,?)

As a workaround I increased the width of the TYPE_NAME column in COLUMNS_V2 table of HMS DB which allowed me to create tables in hive.

But in impala, the table creation fails in the analysis phase
HiveServer2Error: AnalysisException: Type of column 'xxx' exceeds maximum type length of 4000 characters

I see that the MAX_TYPE_NAME_LENGTH is configured to 4000 characters which is causing the problem. Could you please remove this hard limit or make it configurable

https://github.com/cloudera/Impala/blob/cdh5.7.0-release/fe/src/test/java/com/cloudera/impala/analysis/AnalyzeDDLTest.java#L1416-L1418