Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-6114

SQL join selection should be done in planner, not in expansion to PTransform

Details

    • Improvement
    • Status: Triage Needed
    • P2
    • Resolution: Fixed
    • None
    • 2.16.0
    • dsl-sql
    • None

    Description

      Currently Beam SQL joins all go through a single physical operator which has a single PTransform that does all join algorithms based on properties of its input PCollections as well as the relational algebra.

      A first step is to make the needed information part of the relational algebra, so it can choose a PTransform based on that, and the PTransforms can be simpler.

      Second step is to have separate (physical) relational operators for different join algorithms.

      Attachments

        Activity

          People

            rahul8383 Rahul Patwari
            kenn Kenneth Knowles
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 20m
                4h 20m