Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6797

DataNode logs wrong layoutversion during upgrade

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.1
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Before upgrade, data node version was -55. The new data node version remained at -55. During upgrade we got he following messages:

      2014-07-15 12:59:55,253 INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -56
      ...
      ...
      ...
      ...
      2014-07-15 12:59:56,479 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrading block pool storage directory /hadoop/1/data1/current/BP-825373266-xx.xxx.xxx.xx-1379095203239.
         old LV = -55; old CTime = 1402508907789.
         new LV = -56; new CTime = 1405453914270
      2014-07-15 13:00:07,697 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrade of block pool BP-825373266-xx.xxx.xxx.xx-1379095203239 at /hadoop/12/data1/current/BP-825373266-xx.xxx.xxx.xx-1379095203239 is complete
      2014-07-15 13:00:07,839 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=859725752;bpid=BP-825373266-xx.xxx.xxx.xx-1379095203239;lv=-55;nsInfo=lv=-56;cid=CID-303ee504-e03c-4a5e-bc59-2b275b308152;nsid=859725752;c=1405453914270;bpid=BP-82537326
      6-xx.xxx.xxx.xx-1379095203239;dnuuid=b1011b87-d7cd-48ce-92cc-f7cca0e8cbae
      

      after upgrade completing, restart of DN still shows message regarding version difference:

      INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -56
      

      This causes confusion to the operators as if upgrade did not succeed since data node's layout version is not updated to the "new LV" value

      Actually name node's layout version is displayed as the "new LV" value.
      Since the data node and name node layout versions are separate now, the new data node layout version should be shown as the “new LV”.

      Thanks to Enrique Flores who found and reported this issue.

        Attachments

        1. HDFS-6797.patch
          1 kB
          Benoy Antony

          Activity

            People

            • Assignee:
              benoyantony Benoy Antony
              Reporter:
              benoyantony Benoy Antony
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: