Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
-
Fix Version/s: Impala 2.9.0
-
Component/s: Frontend
-
Target Version:
Description
When creating table with complex datatype having more than 4000 characters in hive we received the following error
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Add req_V2` (`CD_ID`,`COMMENT`,`COLUMN_NAME`,`TYPE_NAME`,`INTEGER_IDX`) VALUES (?,?,?,?,?)
As a workaround I increased the width of the TYPE_NAME column in COLUMNS_V2 table of HMS DB which allowed me to create tables in hive.
But in impala, the table creation fails in the analysis phase
HiveServer2Error: AnalysisException: Type of column 'xxx' exceeds maximum type length of 4000 characters
I see that the MAX_TYPE_NAME_LENGTH is configured to 4000 characters which is causing the problem. Could you please remove this hard limit or make it configurable