Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-362

FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In FSEditLog,

      • long and short are first converted to String and are further converted to UTF8
      • For some non-array items, it first create an ArrayWritable object to hold all the items and then writes the ArrayWritable object.

      These result creating many intermediate objects which affects Namenode CPU performance and Namenode restart.

      1. HDFS-362.1.patch
        20 kB
        Uma Maheswara Rao G
      2. HDFS-362.2.patch
        25 kB
        Uma Maheswara Rao G
      3. HDFS-362.2b.patch
        30 kB
        Uma Maheswara Rao G
      4. HDFS-362.2c.patch
        32 kB
        Uma Maheswara Rao G
      5. HDFS-362.2d.patch
        33 kB
        Uma Maheswara Rao G
      6. HDFS-362.2d.patch
        33 kB
        Uma Maheswara Rao G
      7. HDFS-362.2e.patch
        38 kB
        Uma Maheswara Rao G
      8. HDFS-362.patch
        18 kB
        Uma Maheswara Rao G

        Issue Links

          Activity

            People

            • Assignee:
              Uma Maheswara Rao G
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development