XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.3.0
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The DataNode's UUID (DataStorage.getDatanodeUuid() field) is NULL at the point where the FsDataset object is created (DataNode.initStorage().

      As the DataStorage object is an input to the FsDataset factory method, it is desirable for it to be fully populated with a UUID at this point. In particular, our FsDatasetSpi implementation relies upon the DataNode UUID as a key to access our underlying block storage device.

      This also appears to be a regression compared to Hadoop 1.x - our 1.x FSDatasetInterface plugin has a non-NULL UUID on startup. I haven't fully traced through the code, but I suspect this came from the BPOfferService/BPServiceActor refactoring to support federated namenodes.

      With HDFS-5448, the DataNode is now responsible for generating its own UUID. This greatly simplifies the fix. Move the UUID check/generation in from DataNode.createBPRegistration() to DataNode.initStorage(). This more naturally co-locates UUID generation immediately subsequent to the read of the UUID from the DataStorage properties file.

        private void initStorage(final NamespaceInfo nsInfo) throws IOException {
          // ...
      
            final String bpid = nsInfo.getBlockPoolID();
            //read storage info, lock data dirs and transition fs state if necessary
            storage.recoverTransitionRead(this, bpid, nsInfo, dataDirs, startOpt);
            
            // SUGGESTED NEW PLACE TO CHECK DATANODE UUID
            checkDatanodeUuid();
      
          // ...
        }
      

        Attachments

        1. HDFS-5454.01.patch
          1 kB
          Arpit Agarwal
        2. HDFS-5454.02.patch
          1 kB
          Arpit Agarwal
        3. HDFS-5454.03.patch
          5 kB
          Arpit Agarwal

          Activity

            People

            • Assignee:
              arp Arpit Agarwal
              Reporter:
              sirianni Eric Sirianni
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: