Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6797

DataNode logs wrong layoutversion during upgrade

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.4.1
    • 2.6.0
    • datanode
    • None
    • Reviewed

    Description

      Before upgrade, data node version was -55. The new data node version remained at -55. During upgrade we got he following messages:

      2014-07-15 12:59:55,253 INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -56
      ...
      ...
      ...
      ...
      2014-07-15 12:59:56,479 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrading block pool storage directory /hadoop/1/data1/current/BP-825373266-xx.xxx.xxx.xx-1379095203239.
         old LV = -55; old CTime = 1402508907789.
         new LV = -56; new CTime = 1405453914270
      2014-07-15 13:00:07,697 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrade of block pool BP-825373266-xx.xxx.xxx.xx-1379095203239 at /hadoop/12/data1/current/BP-825373266-xx.xxx.xxx.xx-1379095203239 is complete
      2014-07-15 13:00:07,839 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=859725752;bpid=BP-825373266-xx.xxx.xxx.xx-1379095203239;lv=-55;nsInfo=lv=-56;cid=CID-303ee504-e03c-4a5e-bc59-2b275b308152;nsid=859725752;c=1405453914270;bpid=BP-82537326
      6-xx.xxx.xxx.xx-1379095203239;dnuuid=b1011b87-d7cd-48ce-92cc-f7cca0e8cbae
      

      after upgrade completing, restart of DN still shows message regarding version difference:

      INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -56
      

      This causes confusion to the operators as if upgrade did not succeed since data node's layout version is not updated to the "new LV" value

      Actually name node's layout version is displayed as the "new LV" value.
      Since the data node and name node layout versions are separate now, the new data node layout version should be shown as the “new LV”.

      Thanks to ehf who found and reported this issue.

      Attachments

        1. HDFS-6797.patch
          1 kB
          Benoy Antony

        Activity

          People

            benoyantony Benoy Antony
            benoyantony Benoy Antony
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: