Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
A user generated a bulk import file with illegal data. After re-generating the file, they thought they could just move the file into HDFS with the new name. Unfortunately, the block cache remembered some of the data, which caused the data at the block boundaries to be corrupt.
One possible solution is to clear the block cache when an IOException occurs on a read.