Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4462

2NN will fail to checkpoint after an HDFS upgrade from a pre-federation version of HDFS

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.7
    • Component/s: namenode
    • Labels:
      None

      Description

      The 2NN currently has logic to detect when its on-disk FS metadata needs an upgrade with respect to the NN's metadata (i.e. the layout versions are different) and in this case it will proceed with the checkpoint despite storage signatures not matching precisely if the BP ID and Cluster ID do match exactly. However, in situations where we're upgrading from versions of HDFS prior to federation, which had no BP IDs or Cluster IDs, checkpoints will always fail with an error like the following:

      13/01/31 17:02:25 ERROR namenode.SecondaryNameNode: checkpoint: Inconsistent checkpoint fields.
      LV = -40 namespaceID = 403832480 cTime = 1359680537192 ; clusterId = CID-0df6ff22-1165-4c7d-9630-429972a7737c ; blockpoolId = BP-1520616013-172.21.3.106-1359680537136.
      Expecting respectively: -19; 403832480; 0; ; .
      

        Attachments

        1. HDFS-4462.patch
          9 kB
          Aaron Myers
        2. HDFS-4462.patch
          9 kB
          Aaron Myers
        3. HDFS-4462.patch
          11 kB
          Aaron Myers
        4. HDFS-4462.patch
          8 kB
          Aaron Myers
        5. HDFS-4462.patch
          6 kB
          Aaron Myers

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment