Pig
  1. Pig
  2. PIG-2712

Pig does not call OutputCommitter.abortJob() on the underlying OutputFormat

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.3
    • Fix Version/s: 0.11, 0.10.1
    • Component/s: None
    • Labels:
      None
    1. PIG-2712-0.patch
      2 kB
      Daniel Dai
    2. PIG-2712-1-branch10.patch
      27 kB
      Rohini Palaniswamy
    3. PIG-2712-1-trunk.patch
      27 kB
      Rohini Palaniswamy

      Issue Links

        Activity

        Hide
        Daniel Dai added a comment -

        Yes, we are not calling abortJob. Actually abortJob does not exist in 20.2. We shall add the hook now.

        Show
        Daniel Dai added a comment - Yes, we are not calling abortJob. Actually abortJob does not exist in 20.2. We shall add the hook now.
        Hide
        Daniel Dai added a comment -

        Attach a draft patch.

        Show
        Daniel Dai added a comment - Attach a draft patch.
        Hide
        Rohini Palaniswamy added a comment -

        Daniel,

        The signature of abortJob is

           public void abortJob(JobContext jobContext, JobStatus.State state)
        

        The method API and the reflection invocation in the patch need to be corrected.

        Show
        Rohini Palaniswamy added a comment - Daniel, The signature of abortJob is public void abortJob(JobContext jobContext, JobStatus.State state) The method API and the reflection invocation in the patch need to be corrected.
        Hide
        Francis Liu added a comment -

        I'm going to take a crack at this patch.

        Show
        Francis Liu added a comment - I'm going to take a crack at this patch.
        Hide
        Francis Liu added a comment -

        Just spoke with Rohini offline, she has a patch for this. Which just needs to be tested on 0.20.2.

        Show
        Francis Liu added a comment - Just spoke with Rohini offline, she has a patch for this. Which just needs to be tested on 0.20.2.
        Hide
        Rohini Palaniswamy added a comment -

        Tested with 20.2 jars in classpath and invoking cleanupJob. It does not throw any error for JobStatus.

        Show
        Rohini Palaniswamy added a comment - Tested with 20.2 jars in classpath and invoking cleanupJob. It does not throw any error for JobStatus.
        Hide
        Francis Liu added a comment -

        +1 (non-binding), I've tested the 0.10 patch on hadoop-0.23 and verified it fixes HCATALOG-451.

        Show
        Francis Liu added a comment - +1 (non-binding), I've tested the 0.10 patch on hadoop-0.23 and verified it fixes HCATALOG-451 .
        Hide
        Alan Gates added a comment -

        Patch looks good. I'm running the tests and will commit it if passes.

        Show
        Alan Gates added a comment - Patch looks good. I'm running the tests and will commit it if passes.
        Hide
        Alan Gates added a comment -

        Committed on trunk. I'll test it on 0.10 next.

        Show
        Alan Gates added a comment - Committed on trunk. I'll test it on 0.10 next.
        Hide
        Alan Gates added a comment -

        branch10 patch checked into branch-10.

        Show
        Alan Gates added a comment - branch10 patch checked into branch-10.

          People

          • Assignee:
            Rohini Palaniswamy
            Reporter:
            Francis Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development