Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: tez
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After months of development, I feel tez branch reaches to a point we can merge back into trunk. The merge will not introduce any known regression, that is:
      1. All existing unit tests pass in mr mode
      2. All existing e2e tests pass in mr mode
      3. No backward-incompitable changes

      The tez branch is stable enough, that is:
      1. All migrated unit tests pass in tez mode
      2. Vast majority of e2e tests pass in tez mode, minor number of failures are properly investigated

      Ongoing tez work can continue after the merge. It is unlikely major changes to other part of Pig code will be involved going forward.

      Known limitations for Pig on tez at this moment:
      1. Some unit tests are not ported (PIG-3840)
      2. Several operators are missing: native, merge-sparse join, collected cogroup
      3. auto-parallelism is not in (PIG-3846)

      1. PIG-3970-1.patch
        1.75 MB
        Daniel Dai
      2. PIG-3970-1.no_space_change.patch
        1.27 MB
        Daniel Dai
      3. PIG-3970-final.patch
        1.70 MB
        Daniel Dai

        Activity

        Hide
        Daniel Dai added a comment -
        Show
        Daniel Dai added a comment - RB link: https://reviews.apache.org/r/22058/
        Hide
        Daniel Dai added a comment -

        With Cheolsoo and Rohini's +1, I am going to commit it soon.

        Show
        Daniel Dai added a comment - With Cheolsoo and Rohini's +1, I am going to commit it soon.
        Hide
        Daniel Dai added a comment -

        There is one minor conflict with trunk. Attach PIG-3970-final.patch for the final patch I am going to commit.

        Show
        Daniel Dai added a comment - There is one minor conflict with trunk. Attach PIG-3970 -final.patch for the final patch I am going to commit.
        Hide
        Daniel Dai added a comment -

        Patch committed to trunk. Thanks Cheolsoo, Rohini for review!

        Show
        Daniel Dai added a comment - Patch committed to trunk. Thanks Cheolsoo, Rohini for review!

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Daniel Dai
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development