Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
This logic would attempt to create the data necessary for each field by passing through the levels once for each field. it is expected that due to once we can put SIMD/bitmap code in place this will perform better for nested data with shallow nesting, but due to repetitive computation might perform worse for deep nested that include List-types. The SIMD/bitmap enhancements are covered in: ARROW-9985
At a high level the logic would be structured as:
for each field:
for each rep/def level entry:
update null bitmask and offsets.
Attachments
Issue Links
- links to