Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2759

Pre-allocate HDFS edit log files after writing version number

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: ha, namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS-2709 it was discovered that there's a potential race wherein edits log files are pre-allocated before the version number is written into the header of the file. This can cause the NameNode to read an invalid HDFS layout version, and hence fail to read the edit log file. We should write the header, then pre-allocate the rest of the file after this point.

      1. HDFS-2759.patch
        0.9 kB
        Aaron T. Myers
      2. HDFS-2759.patch
        3 kB
        Aaron T. Myers
      3. HDFS-2759.patch
        3 kB
        Aaron T. Myers

        Issue Links

          Activity

          Allen Wittenauer made changes -
          Affects Version/s 2.0.0-alpha [ 12320353 ]
          Affects Version/s 0.24.0 [ 12317653 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Jeff Hammerbacher made changes -
          Link This issue is related to HDFS-2709 [ HDFS-2709 ]
          Todd Lipcon made changes -
          Fix Version/s 2.0.1 [ 12321440 ]
          Fix Version/s 2.0.0 [ 12320353 ]
          Todd Lipcon made changes -
          Fix Version/s 2.0.0 [ 12320353 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Arun C Murthy made changes -
          Fix Version/s 3.0.0 [ 12320356 ]
          Fix Version/s 0.24.0 [ 12317653 ]
          Aaron T. Myers made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.24.0 [ 12317653 ]
          Resolution Fixed [ 1 ]
          Aaron T. Myers made changes -
          Attachment HDFS-2759.patch [ 12512335 ]
          Aaron T. Myers made changes -
          Attachment HDFS-2759.patch [ 12511956 ]
          Aaron T. Myers made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.24.0 [ 12317653 ]
          Affects Version/s HA branch (HDFS-1623) [ 12317568 ]
          Target Version/s 0.24.0 [ 12317653 ]
          Aaron T. Myers made changes -
          Attachment HDFS-2759.patch [ 12511911 ]
          Aaron T. Myers made changes -
          Parent HDFS-1623 [ 12498318 ]
          Issue Type Sub-task [ 7 ] Bug [ 1 ]
          Aaron T. Myers made changes -
          Description In HDFS-2709 it was discovered that there's a potential race wherein edits log files are pre-allocated before the version number is written into the header of the file. This can cause the standby to read an invalid version. We should write the header, then pre-allocate the rest of the file after this point. In HDFS-2709 it was discovered that there's a potential race wherein edits log files are pre-allocated before the version number is written into the header of the file. This can cause the NameNode to read an invalid HDFS layout version, and hence fail to read the edit log file. We should write the header, then pre-allocate the rest of the file after this point.
          Aaron T. Myers made changes -
          Field Original Value New Value
          Component/s ha [ 12316609 ]
          Aaron T. Myers created issue -

            People

            • Assignee:
              Aaron T. Myers
              Reporter:
              Aaron T. Myers
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development