[BEAM-7301] Beam transforms reorder fields

• Type: Sub-task
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-java-core
• Labels: Clarified
• Assignee: Unassigned
• Reporter: Reuven Lax
• Created: 14/May/19 14:58
• Updated: 03/Jun/22 21:14

Description

Currently transforms such as Select, DropFields, RenameFields, and AddFields can create schemas with unexpected order. The problem is that FieldAccessDescriptor stores top-level fields and nested fields separately, so there's no way to tell the relative order between them. To fix this we should refactor FieldAccessDescriptor: instead of storing these separately it should store a single list, where each item in the list might optionally have a nested FieldAccessDescriptor.

Expected behavior from the transforms:

   DropFields: preserves order in original schema

   RenameFields: preserves order in original schema

   AddFields: adds fields in order specified. If multiple nested fields are selected, the first reference to the top field wins (e.g. adding "a.b", "c", "a.d" results in adding a before c.

  Select: Select fields in the order specified.