Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14509

DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x

    XMLWordPrintableJSON

    Details

      Description

      According to the doc, if we want to upgrade cluster from 2.x to 3.x, we need upgrade NN first. And there will be a intermediate state that NN is 3.x and DN is 2.x. At that moment, if a client reads (or writes) a block, it will get a block token from NN and then deliver the token to DN who can verify the token. But the verification in the code now is :

      BlockTokenSecretManager.java
      public void checkAccess(...)
      {
          ...
          id.readFields(new DataInputStream(new ByteArrayInputStream(token.getIdentifier())));
          ...
          if (!Arrays.equals(retrievePassword(id), token.getPassword())) {
            throw new InvalidToken("Block token with " + id.toString()
                + " doesn't have the correct token password");
          }
      }
      

      And retrievePassword(id) is:

       
      public byte[] retrievePassword(BlockTokenIdentifier identifier)
      {
          ...
          return createPassword(identifier.getBytes(), key.getKey());
      }
      

      So, if NN's identifier add new fields, DN will lose the fields and compute wrong password.

        Attachments

        1. HDFS-14509-001.patch
          21 kB
          Brahma Reddy Battula
        2. HDFS-14509-002.patch
          5 kB
          Konstantin Shvachko
        3. HDFS-14509-003.patch
          5 kB
          Yuxuan Wang
        4. HDFS-14509-branch-2.001.patch
          5 kB
          Yuxuan Wang

          Issue Links

            Activity

              People

              • Assignee:
                John Smith Yuxuan Wang
                Reporter:
                John Smith Yuxuan Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: