Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3429

DataNode reads checksums even if client does not need them

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: datanode, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, even if the client does not want to verify checksums, the datanode reads them anyway and sends them over the wire. This means that performance improvements like HBase's application-level checksums don't have much benefit when reading through the datanode, since the DN is still causing seeks into the checksum file.

      (Credit goes to Dhruba for discovering this - filing on his behalf)

        Attachments

        1. hdfs-3429.txt
          34 kB
          Todd Lipcon
        2. hdfs-3429-0.20.2.patch
          5 kB
          LiuLei
        3. hdfs-3429.txt
          34 kB
          Todd Lipcon
        4. hdfs-3429.txt
          21 kB
          Todd Lipcon
        5. hdfs-3429-0.20.2.patch
          24 kB
          LiuLei
        6. hdfs-3429.txt
          19 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                37 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: