Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8058

Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HDFS-7285
    • Fix Version/s: HDFS-7285
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA is to use BlockInfo[] blocks for both striped and contiguous blocks in INodeFile.

      Currently FileWithStripedBlocksFeature keeps separate list for striped blocks, and the methods there duplicate with those in INodeFile, and current code need to judge isStriped then do different things. Also if file is striped, the blocks in INodeFile occupy a reference memory space.
      These are not necessary, and we can use the same blocks to make code more clear.

      I keep FileWithStripedBlocksFeature as empty for follow use: I will file a new JIRA to move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile, since ideally they are the same for all striped blocks in a file, and store them in block will waste NN memory.

        Attachments

        1. HDFS-8058.001.patch
          44 kB
          Yi Liu
        2. HDFS-8058.002.patch
          46 kB
          Yi Liu
        3. HDFS-8058-HDFS-7285.003.patch
          45 kB
          Zhe Zhang
        4. HDFS-8058-HDFS-7285.004.patch
          48 kB
          Zhe Zhang
        5. HDFS-8058-HDFS-7285.005.patch
          66 kB
          Zhe Zhang
        6. HDFS-8058-HDFS-7285.006.patch
          68 kB
          Zhe Zhang
        7. HDFS-8058-HDFS-7285.007.patch
          70 kB
          Zhe Zhang
        8. HDFS-8058-HDFS-7285.008.patch
          74 kB
          Zhe Zhang
        9. HDFS-8058-HDFS-7285.009.patch
          75 kB
          Zhe Zhang
        10. HDFS-8058-HDFS-7285.010.patch
          75 kB
          Zhe Zhang

          Issue Links

            Activity

              People

              • Assignee:
                zhz Zhe Zhang
                Reporter:
                hitliuyi Yi Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: