Hadoop Common
  1. Hadoop Common
  2. HADOOP-6559

The RPC client should try to re-login when it detects that the TGT expired

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently while making RPC calls, the client will throw an exception if the client is not able to use the TGT (expired or timedout). This could be improved - it could catch the exception and try doing a re-login.

      1. h-6559.1.patch
        5 kB
        Devaraj Das
      2. h-6559.2.patch
        5 kB
        Devaraj Das
      3. h-6559.3.patch
        6 kB
        Devaraj Das
      4. h-6559.5.patch
        7 kB
        Devaraj Das
      5. h-6559.6.bp20.patch
        7 kB
        Devaraj Das
      6. h-6559.6.patch
        7 kB
        Devaraj Das

        Activity

        Hide
        Devaraj Das added a comment -

        The attached patch:
        1) The RPC client will try to do re-login when it encounters SaslException. The max #tries is 3 with a 5 seconds sleep between the trials.
        2) Adds a method in the UGI class to do with relogin, and that uses the same keytab/principal that login was originally called with.
        3) The relogin meth keeps track of when it was last run (so as to avoid a flood of such calls, made from different contexts).

        Show
        Devaraj Das added a comment - The attached patch: 1) The RPC client will try to do re-login when it encounters SaslException. The max #tries is 3 with a 5 seconds sleep between the trials. 2) Adds a method in the UGI class to do with relogin, and that uses the same keytab/principal that login was originally called with. 3) The relogin meth keeps track of when it was last run (so as to avoid a flood of such calls, made from different contexts).
        Hide
        Devaraj Das added a comment -

        This is a better version of the same patch.

        Show
        Devaraj Das added a comment - This is a better version of the same patch.
        Hide
        Devaraj Das added a comment -

        Fixes to ensure that re-login only happens if the client did a keytab login earlier. Also, overwrites the ticket in the remoteId object with the new ticket that is created with the new Subject.

        Show
        Devaraj Das added a comment - Fixes to ensure that re-login only happens if the client did a keytab login earlier. Also, overwrites the ticket in the remoteId object with the new ticket that is created with the new Subject.
        Hide
        Kan Zhang added a comment -

        saslRpcClient.saslConnect() needs to be in doAs() block, whereas new SaslRpcClient() may be not.

        no need to return SaslRpcClient as it is a member field.

        please make sure your change is compatible with HADOOP-6543 (i.e., I'll eventually make saslRpcClient.saslConnect() return boolean based on whether we need to fall back to simple auth).

        you need to re-throw SaslException when the retry block is not entered.

        Show
        Kan Zhang added a comment - saslRpcClient.saslConnect() needs to be in doAs() block, whereas new SaslRpcClient() may be not. no need to return SaslRpcClient as it is a member field. please make sure your change is compatible with HADOOP-6543 (i.e., I'll eventually make saslRpcClient.saslConnect() return boolean based on whether we need to fall back to simple auth). you need to re-throw SaslException when the retry block is not entered.
        Hide
        Owen O'Malley added a comment -

        I'd suggest instead making the relogin a non-static method on the UGI and have it pass the Subject into the LoginContext. That will mean that it will keep the same UGI and Subject and you can have the retry code inside of the doAs instead of outside of it.

        Show
        Owen O'Malley added a comment - I'd suggest instead making the relogin a non-static method on the UGI and have it pass the Subject into the LoginContext. That will mean that it will keep the same UGI and Subject and you can have the retry code inside of the doAs instead of outside of it.
        Hide
        Devaraj Das added a comment -

        Ok, here is the patch that addresses the comments Kan/Owen had.

        Show
        Devaraj Das added a comment - Ok, here is the patch that addresses the comments Kan/Owen had.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12435829/h-6559.5.patch
        against trunk revision 909806.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435829/h-6559.5.patch against trunk revision 909806. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/359/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        Attaching a patch that fixes the warnings. I also simplified the patch to not do retrials for the re-login but instead the re-login is attempted once (even for a fresh login the attempt is made only once..).

        Show
        Devaraj Das added a comment - Attaching a patch that fixes the warnings. I also simplified the patch to not do retrials for the re-login but instead the re-login is attempted once (even for a fresh login the attempt is made only once..).
        Hide
        Owen O'Malley added a comment -

        +1

        Show
        Owen O'Malley added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12435847/h-6559.6.patch
        against trunk revision 909806.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435847/h-6559.6.patch against trunk revision 909806. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/360/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        This patch is relevant only to the kerberos setup where i manually tested it. We currently don't have any framework for kerberos junit tests and hence not adding any test here.

        Show
        Devaraj Das added a comment - This patch is relevant only to the kerberos setup where i manually tested it. We currently don't have any framework for kerberos junit tests and hence not adding any test here.
        Hide
        Devaraj Das added a comment -

        I just committed this.

        Show
        Devaraj Das added a comment - I just committed this.
        Hide
        Devaraj Das added a comment -

        Patch for Y!20. Not for commit here.

        Show
        Devaraj Das added a comment - Patch for Y!20. Not for commit here.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #168 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/168/)
        . Makes the RPC client automatically re-login when the SASL connection setup fails. This is applicable only to keytab based logins. Contributed by Devaraj Das.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #168 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/168/ ) . Makes the RPC client automatically re-login when the SASL connection setup fails. This is applicable only to keytab based logins. Contributed by Devaraj Das.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #250 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/250/)
        . Makes the RPC client automatically re-login when the SASL connection setup fails. This is applicable only to keytab based logins. Contributed by Devaraj Das.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #250 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/250/ ) . Makes the RPC client automatically re-login when the SASL connection setup fails. This is applicable only to keytab based logins. Contributed by Devaraj Das.

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development