[BEAM-7268] Make external sorter Hadoop free

• Type: Improvement
• Status: Triage Needed
• Priority: P3
• Resolution: Fixed
• Affects Version/s: 2.13.0
• Fix Version/s: 2.14.0
• Component/s: extensions-java-sorter
• Labels: Done
• Assignee: Neville Li
• Reporter: Neville Li
• Created: 10/May/19 18:06
• Updated: 13/Apr/23 10:55
• Resolved: 14/Jun/19 15:27

Description

Right now the Java sorter extension depends on Hadoop SequenceFile for external sort. It'll be nice to re-implement it without the dependency to avoid conflicts.

Pull request adds a new option to switch to native sorting, the default behavior remains Hadoop sorting.

    final BufferedExternalSorter.Options sorterOpts =
        BufferedExternalSorter
            .options()
            .withExternalSorterType(ExternalSorter.Options.SorterType.HADOOP);

...

        .apply(SortValues.create(sorterOpts));

Issue Links

• links to: GitHub Pull Request #8552