Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8498

Blocks can be committed with wrong size

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When an IBR for a UC block arrives, the NN updates the expected location's block and replica state only if it's on an unexpected storage for an expected DN. If it's for an expected storage, only the genstamp is updated. When the block is committed, and the expected locations are verified, only the genstamp is checked. The size is not checked but it wasn't updated in the expected locations anyway.

      A faulty client may misreport the size when committing the block. The block is effectively corrupted. If the NN issues replications, the received IBR is considered corrupt, the NN invalidates the block, immediately issues another replication. The NN eventually realizes all the original replicas are corrupt after full BRs are received from the original DNs.

        Attachments

        1. HDFS-8498.000.patch
          10 kB
          Jing Zhao
        2. HDFS-8498.001.patch
          11 kB
          Jing Zhao
        3. HDFS-8498.branch-2.001.patch
          13 kB
          Jing Zhao
        4. HDFS-8498.branch-2.7.001.patch
          11 kB
          Wei-Chiu Chuang
        5. HDFS-8498.branch-2.patch
          13 kB
          Jing Zhao
        6. HDFS-8498-branch-2.7.002.patch
          10 kB
          Konstantin Shvachko

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                29 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: