Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-718

configuration parameter to prevent accidental formatting of HDFS filesystem

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Environment:

      Any

    • Hadoop Flags:
      Reviewed

      Description

      Currently, any time the NameNode is not running, an HDFS filesystem will accept the 'format' command, and will duly format itself. There are those of us who have multi-PB HDFS filesystems who are really quite uncomfortable with this behavior. There is "Y/N" confirmation in the format command, but if the formatter genuinely believes themselves to be doing the right thing, the filesystem will be formatted.

      This patch adds a configuration parameter to the namenode, dfs.namenode.support.allowformat, which defaults to "true," the current behavior: always allow formatting if the NameNode is down or some other process is not holding the namenode lock. But if dfs.namenode.support.allowformat is set to "false," the NameNode will not allow itself to be formatted until this config parameter is changed to "true".

      The general idea is that for production HDFS filesystems, the user would format the HDFS once, then set dfs.namenode.support.allowformat to "false" for all time.

      The attached patch was generated against trunk and +1's on my test machine. We have a 0.20 version that we are using in our cluster as well.

      1. HDFS-718.patch.txt
        7 kB
        Andrew Ryan
      2. HDFS-718.patch-2.txt
        8 kB
        Andrew Ryan
      3. HDFS-718-3.patch
        11 kB
        Jakob Homan
      4. HDFS-718-4.patch
        9 kB
        Jakob Homan
      5. HDFS-718-5.patch
        9 kB
        Jakob Homan

        Activity

        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.22.0 [ 12314241 ]
        Resolution Fixed [ 1 ]
        Jakob Homan made changes -
        Attachment HDFS-718-5.patch [ 12459768 ]
        Jakob Homan made changes -
        Attachment HDFS-718-4.patch [ 12459764 ]
        Jakob Homan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jakob Homan made changes -
        Attachment HDFS-718-3.patch [ 12454051 ]
        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        dhruba borthakur made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        dhruba borthakur made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Andrew Ryan made changes -
        Attachment HDFS-718.patch-2.txt [ 12425369 ]
        dhruba borthakur made changes -
        Assignee Andrew Ryan [ andrewr ]
        Andrew Ryan made changes -
        Attachment HDFS-718.patch.txt [ 12422742 ]
        Andrew Ryan made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Andrew Ryan created issue -

          People

          • Assignee:
            Andrew Ryan
            Reporter:
            Andrew Ryan
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development