Details
Sub-task
Description
The DataNode needs to skip checksumming when reading a cached block via non-local reads. This is like HDFS-5182, but for non-short-circuit.
Issue Links
- is related to
-
HDFS-5182
BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid
-
- Open
-
Activity
I guess the TCP and IP checksums are pretty trivial-- basically something similar to summing the value of the bytes. I wish we had more data on the subject of how common network errors were.
Show
Colin Patrick McCabe
added a comment - I guess the TCP and IP checksums are pretty trivial-- basically something similar to summing the value of the bytes. I wish we had more data on the subject of how common network errors were.
Andrew is right about this one... we still want to detect network errors, so let's not skip those checksums in this case.
Show
Colin Patrick McCabe
added a comment - Andrew is right about this one... we still want to detect network errors, so let's not skip those checksums in this case.
| Status | Open [ 1 ] | Resolved [ 5 ] |
| Resolution | Won't Fix [ 2 ] |
I think we still want checksumming for remote reads. TCP / ethernet checksums aren't that strong, and networks are likely a bigger source of errors than disks.