Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2126 Improve Namenode startup time [umbrella task]
  3. HDFS-1071

savenamespace should write the fsimage to all configured fs.name.dir in parallel

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If you have a large number of files in HDFS, the fsimage file is very big. When the namenode restarts, it writes a copy of the fsimage to all directories configured in fs.name.dir. This takes a long time, especially if there are many directories in fs.name.dir. Make the NN write the fsimage to all these directories in parallel.

        Attachments

        1. HDFS-1071.7.patch
          15 kB
          Dmytro Molkov
        2. HDFS-1071.6.patch
          14 kB
          Dmytro Molkov
        3. HDFS-1071.5.patch
          13 kB
          Dmytro Molkov
        4. HDFS-1071.4.patch
          13 kB
          Dmytro Molkov
        5. HDFS-1071.3.patch
          12 kB
          Dmytro Molkov
        6. HDFS-1071.2.patch
          12 kB
          Dmytro Molkov
        7. HDFS-1071.patch
          11 kB
          Dmytro Molkov

          Issue Links

            Activity

              People

              • Assignee:
                dms Dmytro Molkov
                Reporter:
                dhruba dhruba borthakur
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: