[HUDI-2322] Only include meta fields to reorder while preparing dataset for bulk insert

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Labels: pull-request-available
• Assignee: Sagar Sumit
• Reporter: Sagar Sumit
• Created: 19/Aug/21 15:43
• Updated: 20/Aug/21 04:45
• Resolved: 20/Aug/21 04:45

Description

Below filter in `HoodieDatasetBulkInsertHelper` will result in `_hoodie_is_deleted` to be reordered as well even though it is not part of meta columns.

List<Column> originalFields =
        Arrays.stream(rowsWithMetaCols.schema().fields()).filter(field -> !field.name().contains("_hoodie_")).map(f -> new Column(f.name())).collect(Collectors.toList());

    List<Column> metaFields =
        Arrays.stream(rowsWithMetaCols.schema().fields()).filter(field -> field.name().contains("_hoodie_")).map(f -> new Column(f.name())).collect(Collectors.toList());

The fix is to check only for `HoodieRecord.HOODIE_META_COLUMNS_WITH_OPERATION`.

Issue Links

• links to: GitHub Pull Request #3504