Pig
  1. Pig
  2. PIG-4008

Pig code change to enable Tez Local mode

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: tez
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Once pig specifes using local mode and tez framework, it will pass local parameter to tez and enable tez to use local mode.

      1. PIG-4008.patch
        0.9 kB
        Chen He
      2. PIG-4008-v2.patch
        0.8 kB
        Chen He
      3. PIG-4008-3.patch
        0.8 kB
        Daniel Dai
      4. PIG-4008-4.patch
        2 kB
        Daniel Dai

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5m 32s 1 Chen He 11/Jun/14 17:39
          Patch Available Patch Available Resolved Resolved
          54d 7h 33m 1 Daniel Dai 05/Aug/14 01:13
          Resolved Resolved Closed Closed
          108d 5h 44m 1 Daniel Dai 21/Nov/14 05:58
          Daniel Dai made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Daniel Dai made changes -
          Component/s tez [ 12321016 ]
          Daniel Dai made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.14.0 [ 12326954 ]
          Resolution Fixed [ 1 ]
          Hide
          Daniel Dai added a comment -

          Patch committed to trunk. Thanks Chen He!

          Show
          Daniel Dai added a comment - Patch committed to trunk. Thanks Chen He !
          Hide
          Chen He added a comment -

          multiDAG will be considered in TEZ-1376. Thank you Daniel Dai and Cheolsoo Park

          Show
          Chen He added a comment - multiDAG will be considered in TEZ-1376 . Thank you Daniel Dai and Cheolsoo Park
          Daniel Dai made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          Daniel Dai made changes -
          Attachment PIG-4008-4.patch [ 12659704 ]
          Hide
          Daniel Dai added a comment -

          Another update after TEZ-1343 check in. Now tez local mode should work in most cases. Siddharth Seth also point out if Pig submit multiple DAG simultaneously, local mode will fail. But what we have right now is already very useful, so I would like to check this in and improve over time.

          Show
          Daniel Dai added a comment - Another update after TEZ-1343 check in. Now tez local mode should work in most cases. Siddharth Seth also point out if Pig submit multiple DAG simultaneously, local mode will fail. But what we have right now is already very useful, so I would like to check this in and improve over time.
          Daniel Dai made changes -
          Attachment PIG-4008-3.patch [ 12659284 ]
          Daniel Dai made changes -
          Attachment PIG-4008-3.patch [ 12659279 ]
          Daniel Dai made changes -
          Attachment PIG-4008-3.patch [ 12659279 ]
          Hide
          Daniel Dai added a comment -

          A small fix to improve the usability. User don't need to set PIG_CLASSPATH and tez.lib.uris when using tez_local.

          Show
          Daniel Dai added a comment - A small fix to improve the usability. User don't need to set PIG_CLASSPATH and tez.lib.uris when using tez_local.
          Chen He made changes -
          Attachment PIG-4008-v2.patch [ 12656931 ]
          Hide
          Chen He added a comment -

          updated according to latest pig.

          Show
          Chen He added a comment - updated according to latest pig.
          Chen He made changes -
          Link This issue relates to TEZ-684 [ TEZ-684 ]
          Hide
          Chen He added a comment -

          Hi Cheolsoo Park,
          Thank you for assigning this to me.
          Yes, it is, I will link them together.

          Show
          Chen He added a comment - Hi Cheolsoo Park , Thank you for assigning this to me. Yes, it is, I will link them together.
          Cheolsoo Park made changes -
          Assignee Chen He [ airbots ]
          Hide
          Cheolsoo Park added a comment -

          This is great! Thank you for working on this.

          Btw, does this jira depends on TEZ-710? If so, can you link them together?

          Show
          Cheolsoo Park added a comment - This is great! Thank you for working on this. Btw, does this jira depends on TEZ-710 ? If so, can you link them together?
          Chen He made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Chen He made changes -
          Field Original Value New Value
          Attachment PIG-4008.patch [ 12649822 ]
          Chen He created issue -

            People

            • Assignee:
              Chen He
              Reporter:
              Chen He
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development