[HIVE-9500] Support nested structs over 24 levels.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Labels: SerDe TODOC1.2
• Assignee: Aihua Xu
• Reporter: Aihua Xu
• Created: 28/Jan/15 18:11
• Updated: 13/Nov/15 07:40
• Resolved: 12/Feb/15 21:38

Description

Customer has deeply nested avro structure and is receiving the following error when performing queries.

15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting supported for LazySimpleSerde is 23 Unable to work with level 24
Currently we support up to 24 levels of nested structs when hive.serialization.extend.nesting.levels is set to true, while the customers have the requirement to support more than that.

It would be better to make the supported levels configurable or completely removed (i.e., we can support any number of levels).

Issue Links

• is related to: HIVE-12406 HIVE-9500 introduced incompatible change to LazySimpleSerDe public interface
• relates to: HIVE-3253 ArrayIndexOutOfBounds exception for deeply nested structs