Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-362

FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In FSEditLog,

      • long and short are first converted to String and are further converted to UTF8
      • For some non-array items, it first create an ArrayWritable object to hold all the items and then writes the ArrayWritable object.

      These result creating many intermediate objects which affects Namenode CPU performance and Namenode restart.

        Attachments

        1. HDFS-362.2e.patch
          38 kB
          Uma Maheswara Rao G
        2. HDFS-362.2d.patch
          33 kB
          Uma Maheswara Rao G
        3. HDFS-362.2d.patch
          33 kB
          Uma Maheswara Rao G
        4. HDFS-362.2c.patch
          32 kB
          Uma Maheswara Rao G
        5. HDFS-362.2b.patch
          30 kB
          Uma Maheswara Rao G
        6. HDFS-362.2.patch
          25 kB
          Uma Maheswara Rao G
        7. HDFS-362.1.patch
          20 kB
          Uma Maheswara Rao G
        8. HDFS-362.patch
          18 kB
          Uma Maheswara Rao G

          Issue Links

            Activity

              People

              • Assignee:
                umamaheswararao Uma Maheswara Rao G
                Reporter:
                szetszwo Tsz Wo Nicholas Sze
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: