Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1114

Reducing NameNode memory usage by an alternate hash table

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      NameNode uses a java.util.HashMap to store BlockInfo objects. When there are many blocks in HDFS, this map uses a lot of memory in the NameNode. We may optimize the memory usage by a light weight hash table implementation.

        Attachments

        1. GSet20100525.pdf
          105 kB
          Tsz Wo Nicholas Sze
        2. h1114_20100607.patch
          14 kB
          Tsz Wo Nicholas Sze
        3. gset20100608.pdf
          105 kB
          Tsz Wo Nicholas Sze
        4. h1114_20100614b.patch
          24 kB
          Tsz Wo Nicholas Sze
        5. h1114_20100615.patch
          25 kB
          Tsz Wo Nicholas Sze
        6. h1114_20100616b.patch
          27 kB
          Tsz Wo Nicholas Sze
        7. h1114_20100617.patch
          27 kB
          Tsz Wo Nicholas Sze
        8. h1114_20100617b.patch
          27 kB
          Tsz Wo Nicholas Sze
        9. benchmark20100618.patch
          4 kB
          Tsz Wo Nicholas Sze
        10. h1114_20100617b_y0.20.1xx.patch
          27 kB
          Tsz Wo Nicholas Sze
        11. h1114_20100617b2_y0.20.1xx.patch
          27 kB
          Tsz Wo Nicholas Sze
        12. gset20100702.pdf
          107 kB
          Tsz Wo Nicholas Sze
        13. gset20100702.tex
          14 kB
          Tsz Wo Nicholas Sze

          Activity

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: