Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: tez-branch
    • Fix Version/s: tez-branch
    • Component/s: tez
    • Labels:
      None

      Activity

      Hide
      Cheolsoo Park added a comment -

      Assigning to Alex.

      Show
      Cheolsoo Park added a comment - Assigning to Alex.
      Show
      Alex Bain added a comment - https://reviews.apache.org/r/15219/
      Hide
      Alex Bain added a comment -

      Notes from my ReviewBoard post:
      ===========================
      Implement DISTINCT for Pig-on-Tez by providing a (very straightforward) implementation in TezCompiler.java.

      For the moment, this does NOT use two optimizations done in the MRCompiler. We will create a separate JIRA for these optimizations:
      1. A distinct combiner
      2. A combiner optimizer that replaces certain uses of DISTINCT with an algebraic udf

      This patch includes:
      -A unit test in TestTezCompiler.java
      -An e2e test

      Show
      Alex Bain added a comment - Notes from my ReviewBoard post: =========================== Implement DISTINCT for Pig-on-Tez by providing a (very straightforward) implementation in TezCompiler.java. For the moment, this does NOT use two optimizations done in the MRCompiler. We will create a separate JIRA for these optimizations: 1. A distinct combiner 2. A combiner optimizer that replaces certain uses of DISTINCT with an algebraic udf This patch includes: -A unit test in TestTezCompiler.java -An e2e test
      Hide
      Cheolsoo Park added a comment -

      +1.

      Show
      Cheolsoo Park added a comment - +1.
      Hide
      Cheolsoo Park added a comment -

      Committed to tez branch. Thank you Alex!

      Show
      Cheolsoo Park added a comment - Committed to tez branch. Thank you Alex!
      Hide
      Alex Bain added a comment -

      Distinct optimizations pushed to https://issues.apache.org/jira/browse/PIG-3562

      Show
      Alex Bain added a comment - Distinct optimizations pushed to https://issues.apache.org/jira/browse/PIG-3562

        People

        • Assignee:
          Alex Bain
          Reporter:
          Cheolsoo Park
        • Votes:
          0 Vote for this issue
          Watchers:
          2 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development