Hadoop Common
  1. Hadoop Common
  2. HADOOP-9461

JobTracker and NameNode both grant delegation tokens to non-secure clients

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change

      Description

      If one looks at the MAPREDUCE-1516 added logic in JobTracker.java's isAllowedDelegationTokenOp() method, and apply non-secure states of UGI.isSecurityEnabled == false and authMethod == SIMPLE, the return result is true when the intention is false (due to the shorted conditionals).

      This is allowing non-secure JobClients to easily request and use DelegationTokens and cause unwanted errors to be printed in the JobTracker when the renewer attempts to run. Ideally such clients ought to get an error if they request a DT in non-secure mode.

      HDFS in trunk and branch-1 both too have the same problem. Trunk MR (HistoryServer) and YARN are however, unaffected due to a simpler, inlined logic instead of reuse of this faulty method.

      Note that fixing this will break Oozie today, due to the merged logic of OOZIE-734. Oozie will require a fix as well if this is to be fixed in branch-1. As a result, I'm going to mark this as an Incompatible Change.

        Activity

        Hide
        Harsh J added a comment -

        Not an issue on trunk/branch-2.

        Show
        Harsh J added a comment - Not an issue on trunk/branch-2.
        Hide
        Daryn Sharp added a comment -

        ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as mr token

        This is the issue I expected to see - which is resolved by YARN-320. It's only an issue if security is enabled and a job needs to submit a sub-job more than 1d later. Even though an insecure JT is issuing tokens, an insecure client won't send the token, and even if it does, the insecure JT tells the client to switch back to SIMPLE.

        If you want to backport YARN-320, it shouldn't be too hard. The suboptimal workaround for secure clusters is to increase the MR token's expiration to something like 1w so renewal isn't necessary.

        WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN

        This is odd because it found the class to get the mismatched renewer error. Maybe I'm misremembering the token types in 1.x.

        Show
        Daryn Sharp added a comment - ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as mr token This is the issue I expected to see - which is resolved by YARN-320 . It's only an issue if security is enabled and a job needs to submit a sub-job more than 1d later. Even though an insecure JT is issuing tokens, an insecure client won't send the token, and even if it does, the insecure JT tells the client to switch back to SIMPLE. If you want to backport YARN-320 , it shouldn't be too hard. The suboptimal workaround for secure clusters is to increase the MR token's expiration to something like 1w so renewal isn't necessary. WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN This is odd because it found the class to get the mismatched renewer error. Maybe I'm misremembering the token types in 1.x.
        Hide
        Harsh J added a comment -

        Thanks for taking a look Daryn. Yes the error exactly is:

        ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as mr token
        WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN
        

        It is non-fatal, but appears at a high level of log, so is annoying/misleading. I'll take a look at YARN-320. Do you think this is also worth fixing for branch-1?

        No, services are expected to return null to a token request if security isn't enabled.

        Thanks for clarifying this - I'm guessing they're currently receiving back a valid token though (at least on my MRv1 tests).

        Show
        Harsh J added a comment - Thanks for taking a look Daryn. Yes the error exactly is: ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client mapred tries to renew a token with renewer specified as mr token WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN It is non-fatal, but appears at a high level of log, so is annoying/misleading. I'll take a look at YARN-320 . Do you think this is also worth fixing for branch-1? No, services are expected to return null to a token request if security isn't enabled. Thanks for clarifying this - I'm guessing they're currently receiving back a valid token though (at least on my MRv1 tests).
        Hide
        Daryn Sharp added a comment -

        This is allowing non-secure JobClients to easily request and use DelegationTokens and cause unwanted errors to be printed in the JobTracker when the renewer attempts to run.

        What is the error message? Is it about the renewer being incorrect? If so, this was resolved in trunk/2/23 via YARN-320.

        Ideally such clients ought to get an error if they request a DT in non-secure mode.

        No, services are expected to return null to a token request if security isn't enabled. The client is supposed to interpret this as no token is required. This pattern is supposed to allow a client to operate against both a secure and insecure cluster.

        Show
        Daryn Sharp added a comment - This is allowing non-secure JobClients to easily request and use DelegationTokens and cause unwanted errors to be printed in the JobTracker when the renewer attempts to run. What is the error message? Is it about the renewer being incorrect? If so, this was resolved in trunk/2/23 via YARN-320 . Ideally such clients ought to get an error if they request a DT in non-secure mode. No, services are expected to return null to a token request if security isn't enabled. The client is supposed to interpret this as no token is required. This pattern is supposed to allow a client to operate against both a secure and insecure cluster.

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development