Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1774

Remove use of INode.parent in Block CRC upgrade

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.15.0
    • None
    • None

    Description

      HADOOP-1687 proposes various reductions in Namenode memory. One of the 'cuts' is to remove 'parent' field in INodes (8 bytes per file). HADOOP-1743 already deprecates INode.getAbsoluteName() method.

      Block CRC Upgrade code still requires this field (blockID -> fileName -> parent -> INode for .crc file). If we don't want to use parent, we need to build INode-to-parent map at the beginning of the upgrade. Note this would increase memory required during upgrade by 50 bytes for each of non-crc files (50% of files). I would estimate this to be around 5% more memory.

      Attachments

        1. HADOOP-1774.patch
          8 kB
          Raghu Angadi
        2. HADOOP-1774.patch
          7 kB
          Raghu Angadi
        3. HADOOP-1774.patch
          7 kB
          Raghu Angadi
        4. HADOOP-1774-correction.patch
          1 kB
          Raghu Angadi

        Activity

          People

            rangadi Raghu Angadi
            rangadi Raghu Angadi
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: