Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4465

Optimize datanode ReplicasMap and ReplicaInfo

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.5-alpha
    • 2.1.0-beta
    • datanode
    • None
    • Reviewed

    Description

      In Hadoop a lot of optimization has been done in namenode data structures to be memory efficient. Similar optimizations are necessary for Datanode process. With the growth in storage per datanode and number of blocks hosted on datanode, this jira intends to optimize long lived ReplicasMap and ReplicaInfo objects.

      Attachments

        1. dn-memory-improvements.patch
          14 kB
          Aaron Myers
        2. HDFS-4465.patch
          9 kB
          Aaron Myers
        3. HDFS-4465.patch
          13 kB
          Aaron Myers
        4. HDFS-4465.patch
          13 kB
          Aaron Myers

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-4461 DirectoryScanner: volume path prefix takes up memory for every block that is scanned

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              atm Aaron Myers
              sureshms Suresh Srinivas
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: