[DRILL-7096] Develop vector for canonical Map<K,V>

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Labels: ready-to-commit
• Assignee: Bohdan Kazydub
• Reporter: Igor Guzenko
• Created: 12/Mar/19 14:59
• Updated: 19/Sep/19 11:03
• Resolved: 03/Sep/19 15:11

Description

Canonical Map<K,V> datatype can be represented using combination of three value vectors:

keysVector - vector for storing keys of each map
valuesVector - vector for storing values of each map
offsetsVector - vector for storing of start indexes of next each map

So it's not very hard to create such Map vector, but there is a major issue with such map representation. It's hard to search maps values by key in such vector, need to investigate some advanced techniques to make such search efficient. Or find other more suitable options to represent map datatype in world of vectors.

After question about maps, Apache Arrow developers responded that for Java they don't have real Map vector, for now they just have logical Map type definition where they define Map like: List< Struct<key:key_type, value:value_type> >. So implementation of value vector would be useful for Arrow too.

Issue Links

• blocks: DRILL-7252 Read Hive map using Dict<K,V> vector
• incorporates: DRILL-2000 Hive generated parquet files with maps show up in drill as map(key value)
• is blocked by: DRILL-7097 Rename MapVector to StructVector
• is related to: DRILL-7359 Add support for DICT type in RowSet Framework
• is related to: DRILL-2000 Hive generated parquet files with maps show up in drill as map(key value)
• links to: Canonical MapVector design doc [WIP]
• links to: GitHub Pull Request #1829
• links to: MapVector design presentation