Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
time is spent at resizing vectors around here or in some other "ensureSize" method
create table t as select named_struct('id',13,'str','string','nest',named_struct('id',12,'str','string','arr',array('value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value','value'))) s; -- go up to 1M rows insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; -- insert into table t select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t union all select * from t; set hive.fetch.task.conversion=none; select count(1) from t; --explain select s .id from t where s .nest .id > 0;
interestingly; the issue is not present:
- for a query not looking into the nested struct
- and in case the struct with the array is at the top level
select count(1) from t; --explain select s .id from t where s -- .nest .id > 0;
Attachments
Issue Links
- is related to
-
HIVE-19043 Vectorization: LazySimpleDeserializeRead fewer fields handling is broken for Complex Types
- Closed
- links to