Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1443 Improve Datanode startup time
  3. HDFS-1447

Make getGenerationStampFromFile() more efficient, so it doesn't reprocess full directory listing for every block

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.20.2
    • None
    • datanode
    • None
    • datanode startup

    Description

      Make getGenerationStampFromFile() more efficient. Currently this routine is called by addToReplicasMap() for every blockfile in the directory tree, and it walks each file's containing directory on every call. There is a simple refactoring that should make it more efficient.

      This work item is one of four sub-tasks for HDFS-1443, Improve Datanode startup time.
      The fix will probably be folded into sibling task HDFS-1446, which is already refactoring the method that calls getGenerationStampFromFile().

      Attachments

        1. HDFS-1447.patch
          7 kB
          Uma Maheswara Rao G
        2. Test_HDFS_1447_NotForCommitt.java.patch
          6 kB
          Uma Maheswara Rao G

        Activity

          People

            mattf Matthew Foley
            mattf Matthew Foley
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated: