Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: tez-branch
    • Fix Version/s: tez-branch
    • Component/s: tez
    • Labels:
      None

      Description

      Skewed join in Tez can be implemented similarly to order-by (PIG-3634).

      1. PIG-3644-2.patch
        93 kB
        Cheolsoo Park
      2. PIG-3644-1.patch
        94 kB
        Cheolsoo Park

        Activity

        Hide
        Cheolsoo Park added a comment -

        Attaching the 1st patch. The RB link is-
        https://reviews.apache.org/r/16860/

        Show
        Cheolsoo Park added a comment - Attaching the 1st patch. The RB link is- https://reviews.apache.org/r/16860/
        Hide
        Cheolsoo Park added a comment -

        Rebased to tez head and removed some unused methods in POPartitionRearrangeTez. I will commit this patch shortly.

        Regarding the unit test TestSkewedJoin, I am running into OOM errors inside tez mini-cluster. I will commit the patch and follow up in a separate jira.

        Show
        Cheolsoo Park added a comment - Rebased to tez head and removed some unused methods in POPartitionRearrangeTez. I will commit this patch shortly. Regarding the unit test TestSkewedJoin, I am running into OOM errors inside tez mini-cluster. I will commit the patch and follow up in a separate jira.
        Hide
        Cheolsoo Park added a comment -

        Committed to tez branch. Thank you Daniel for the review!

        Show
        Cheolsoo Park added a comment - Committed to tez branch. Thank you Daniel for the review!

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development