Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10410

Does not work specifying the docker client configuration in distributed shell

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2
    • Fix Version/s: None
    • Labels:
      None

      Description

      Job sumbit client side log  

      2020-08-26 12:07:30,756 INFO distributedshell.Client: Got dt for hdfs://at; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:at, Ident: (token for magnum: HDFS_DELEGATION_TOKEN owner=magnum@TEST.COM, renewer=yarn, realUser=, issueDate=1598411249950, maxDate=1599275249950, sequenceNumber=1359401, masterKeyId=461)
      2020-08-26 12:07:30,757 INFO distributedshell.Client: Got dt for hdfs://at; Kind: kms-dt, Service: kms://https@ranger-kms.test.com:9393/kms, Ident: (kms-dt owner=magnum, renewer=yarn, realUser=, issueDate=1598411250569, maxDate=1599016050569, sequenceNumber=2791602, masterKeyId=908)
      2020-08-26 12:07:30,969 INFO util.DockerClientConfigHandler: Token read from Docker client configuration file: Kind: DOCKER_CLIENT_CREDENTIAL_TOKEN, Service: reg.test.com, Ident: (registryUrl: "reg.test.com" applicationId: "application_1593772175687_475926")
       

       

      AM LOG

      20/08/26 12:07:33 INFO distributedshell.ApplicationMaster: Executing with tokens:
      20/08/26 12:07:33 INFO distributedshell.ApplicationMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 475926 cluster_timestamp: 1593772175687 } attemptId: 1 } keyId: 1589057158)
      20/08/26 12:07:33 INFO distributedshell.ApplicationMaster: Kind: kms-dt, Service: kms://https@ranger-kms.test.com:9393/kms, Ident: (kms-dt owner=magnum, renewer=yarn, realUser=, issueDate=1598411250569, maxDate=1599016050569, sequenceNumber=2791602, masterKeyId=908)
      20/08/26 12:07:33 INFO distributedshell.ApplicationMaster: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:at, Ident: (token for magnum: HDFS_DELEGATION_TOKEN owner=magnum@TEST.COM, renewer=yarn, realUser=, issueDate=1598411249950, maxDate=1599275249950, sequenceNumber=1359401, masterKeyId=461)
      20/08/26 12:07:33 INFO impl.NMClientAsyncImpl: Upper bound of the thread pool size is 500
      20/08/26 12:07:33 WARN distributedshell.ApplicationMaster: Timeline service is not enabled
      

       

      On the client side in distributea shell, Client read docker_client_config and put it in credentials instances
      DOCKER_CLIENT_CREDENTIAL_TOKEN is not propagated to the container.
      eventually docker container launch that requires auth token failed. 

       

        Attachments

        1. YARN-10410.002.patch
          2 kB
          KWON BYUNGCHANG
        2. YARN-10410.001.patch
          2 kB
          KWON BYUNGCHANG

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                magnum KWON BYUNGCHANG
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: