Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 1.0.0
    • Component/s: jobtracker
    • Labels:
      None

      Description

      When external systems submit jobs whose tasks need to submit additional jobs (such as oozie/pig), they include their own MR token used to submit the job. The token's renewer may not allow the JT to renew the token. The JT log will include very long SASL/GSSAPI exceptions when the job is submitted. It is also dubious for the JT to renew its token because it renders the expiry as meaningless since the JT will renew its own token until the max lifetime is exceeded.

      After speaking with Owen & Jitendra, the immediate solution is for the JT to not attempt to renew its own tokens.

      1. MAPREDUCE-3475.patch
        3 kB
        Daryn Sharp
      2. MAPREDUCE-3475-1.patch
        3 kB
        Daryn Sharp

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505350/MAPREDUCE-3475.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1348//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505350/MAPREDUCE-3475.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1348//console This message is automatically generated.
        Hide
        Owen O'Malley added a comment -

        Actually, I think this is the wrong direction. In general the JT should renew its own tokens.

        I'd suggest rather that the broken tokens from Oozie not be renewed by not trying to renew any tokens that don't have the proper user as either owner or renewer. That is a much more general fix that will handle a wider set of upstream problems.

        Show
        Owen O'Malley added a comment - Actually, I think this is the wrong direction. In general the JT should renew its own tokens. I'd suggest rather that the broken tokens from Oozie not be renewed by not trying to renew any tokens that don't have the proper user as either owner or renewer. That is a much more general fix that will handle a wider set of upstream problems.
        Hide
        Daryn Sharp added a comment -

        A truly general solution will require touching all components. This patch fixes the issues for MR tokens by checking the token's renewer against the login user.

        Show
        Daryn Sharp added a comment - A truly general solution will require touching all components. This patch fixes the issues for MR tokens by checking the token's renewer against the login user.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505394/MAPREDUCE-3475-1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1355//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505394/MAPREDUCE-3475-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1355//console This message is automatically generated.
        Hide
        Owen O'Malley added a comment -

        Daryn,
        Can you look for a match with either the owner or renewer? They should both be able to do renewals.

        Show
        Owen O'Malley added a comment - Daryn, Can you look for a match with either the owner or renewer? They should both be able to do renewals.
        Hide
        Daryn Sharp added a comment -

        Unfortunately the secret manager only looks at renewer. The owner isn't factored in for some reason...

        Show
        Daryn Sharp added a comment - Unfortunately the secret manager only looks at renewer. The owner isn't factored in for some reason...
        Hide
        Owen O'Malley added a comment -

        Well, that is a mistake. But since the filter matches the current behavior, that is fine. The patch looks fine to me.

        Show
        Owen O'Malley added a comment - Well, that is a mistake. But since the filter matches the current behavior, that is fine. The patch looks fine to me.
        Hide
        Matt Foley added a comment -

        Taking Owen's comment for a +1, I have committed this patch to 1.0.0 and branch-1.
        Leaving the jira open with the expectation that this same fix is needed in v0.22.

        Show
        Matt Foley added a comment - Taking Owen's comment for a +1, I have committed this patch to 1.0.0 and branch-1. Leaving the jira open with the expectation that this same fix is needed in v0.22.

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:

              Development