Pig
  1. Pig
  2. PIG-2712

Pig does not call OutputCommitter.abortJob() on the underlying OutputFormat

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.3
    • Fix Version/s: 0.11, 0.10.1
    • Component/s: None
    • Labels:
      None
    1. PIG-2712-1-trunk.patch
      27 kB
      Rohini Palaniswamy
    2. PIG-2712-1-branch10.patch
      27 kB
      Rohini Palaniswamy
    3. PIG-2712-0.patch
      2 kB
      Daniel Dai

      Issue Links

        Activity

        Francis Liu created issue -
        Francis Liu made changes -
        Field Original Value New Value
        Link This issue is related to HCATALOG-413 [ HCATALOG-413 ]
        Hide
        Daniel Dai added a comment -

        Yes, we are not calling abortJob. Actually abortJob does not exist in 20.2. We shall add the hook now.

        Show
        Daniel Dai added a comment - Yes, we are not calling abortJob. Actually abortJob does not exist in 20.2. We shall add the hook now.
        Hide
        Daniel Dai added a comment -

        Attach a draft patch.

        Show
        Daniel Dai added a comment - Attach a draft patch.
        Daniel Dai made changes -
        Attachment PIG-2712-0.patch [ 12528669 ]
        Rohini Palaniswamy made changes -
        Link This issue relates to HCATALOG-451 [ HCATALOG-451 ]
        Hide
        Rohini Palaniswamy added a comment -

        Daniel,

        The signature of abortJob is

           public void abortJob(JobContext jobContext, JobStatus.State state)
        

        The method API and the reflection invocation in the patch need to be corrected.

        Show
        Rohini Palaniswamy added a comment - Daniel, The signature of abortJob is public void abortJob(JobContext jobContext, JobStatus.State state) The method API and the reflection invocation in the patch need to be corrected.
        Hide
        Francis Liu added a comment -

        I'm going to take a crack at this patch.

        Show
        Francis Liu added a comment - I'm going to take a crack at this patch.
        Hide
        Francis Liu added a comment -

        Just spoke with Rohini offline, she has a patch for this. Which just needs to be tested on 0.20.2.

        Show
        Francis Liu added a comment - Just spoke with Rohini offline, she has a patch for this. Which just needs to be tested on 0.20.2.
        Hide
        Rohini Palaniswamy added a comment -

        Tested with 20.2 jars in classpath and invoking cleanupJob. It does not throw any error for JobStatus.

        Show
        Rohini Palaniswamy added a comment - Tested with 20.2 jars in classpath and invoking cleanupJob. It does not throw any error for JobStatus.
        Rohini Palaniswamy made changes -
        Attachment PIG-2712-1-branch10.patch [ 12543994 ]
        Attachment PIG-2712-1-trunk.patch [ 12543995 ]
        Rohini Palaniswamy made changes -
        Assignee Rohini Palaniswamy [ rohini ]
        Fix Version/s 0.11 [ 12318878 ]
        Fix Version/s 0.10.1 [ 12320547 ]
        Rohini Palaniswamy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Francis Liu added a comment -

        +1 (non-binding), I've tested the 0.10 patch on hadoop-0.23 and verified it fixes HCATALOG-451.

        Show
        Francis Liu added a comment - +1 (non-binding), I've tested the 0.10 patch on hadoop-0.23 and verified it fixes HCATALOG-451 .
        Hide
        Alan Gates added a comment -

        Patch looks good. I'm running the tests and will commit it if passes.

        Show
        Alan Gates added a comment - Patch looks good. I'm running the tests and will commit it if passes.
        Hide
        Alan Gates added a comment -

        Committed on trunk. I'll test it on 0.10 next.

        Show
        Alan Gates added a comment - Committed on trunk. I'll test it on 0.10 next.
        Hide
        Alan Gates added a comment -

        branch10 patch checked into branch-10.

        Show
        Alan Gates added a comment - branch10 patch checked into branch-10.
        Alan Gates made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Rohini Palaniswamy
            Reporter:
            Francis Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development