Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14509

DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      According to the doc, if we want to upgrade cluster from 2.x to 3.x, we need upgrade NN first. And there will be a intermediate state that NN is 3.x and DN is 2.x. At that moment, if a client reads (or writes) a block, it will get a block token from NN and then deliver the token to DN who can verify the token. But the verification in the code now is :

      BlockTokenSecretManager.java
      public void checkAccess(...)
      {
          ...
          id.readFields(new DataInputStream(new ByteArrayInputStream(token.getIdentifier())));
          ...
          if (!Arrays.equals(retrievePassword(id), token.getPassword())) {
            throw new InvalidToken("Block token with " + id.toString()
                + " doesn't have the correct token password");
          }
      }
      

      And retrievePassword(id) is:

       
      public byte[] retrievePassword(BlockTokenIdentifier identifier)
      {
          ...
          return createPassword(identifier.getBytes(), key.getKey());
      }
      

      So, if NN's identifier add new fields, DN will lose the fields and compute wrong password.

      Attachments

        1. HDFS-14509-branch-2.001.patch
          5 kB
          Yuxuan Wang
        2. HDFS-14509-003.patch
          5 kB
          Yuxuan Wang
        3. HDFS-14509-002.patch
          5 kB
          Konstantin Shvachko
        4. HDFS-14509-001.patch
          21 kB
          Brahma Reddy Battula

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            John Smith Yuxuan Wang
            John Smith Yuxuan Wang
            Votes:
            0 Vote for this issue
            Watchers:
            26 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment