[HDDS-6000] Freon datanode chunk validator fails checksum validation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: freon
• Labels: pull-request-available
• Target Version/s: 1.3.0
• Assignee: Attila Doroszlai
• Reporter: Attila Doroszlai
• Created: 17/Nov/21 14:59
• Updated: 01/Dec/21 06:51
• Resolved: 01/Dec/21 06:51

Description

Seems like Freon Datanode Chunk Validator always fails checksum validation:

$ ozone freon dcg -n2 -t1 -p dcg2
...
Successful executions: 2

$ ozone freon dcv -n2 -t1 -p dcg2
...
ERROR freon.BaseFreonGenerator: Error on executing task 1
java.lang.IllegalStateException: Reference (=first) message checksum doesn't match with checksum of chunk
	at org.apache.hadoop.ozone.freon.DatanodeChunkValidator.lambda$validateChunk$4(DatanodeChunkValidator.java:245)
	at com.codahale.metrics.Timer.time(Timer.java:116)
	at org.apache.hadoop.ozone.freon.DatanodeChunkValidator.validateChunk(DatanodeChunkValidator.java:226)
	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.tryNextTask(BaseFreonGenerator.java:182)
	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.taskLoop(BaseFreonGenerator.java:162)
	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.lambda$startTaskRunners$0(BaseFreonGenerator.java:145)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
 100.00% |?????????????????????????????????????????????????????????????????????????????????????????????????????|  2/2 Time: 0:00:04
One ore more freon test is failed.
11/17/21, 2:58:15 PM ===========================================================

-- Timers ----------------------------------------------------------------------
chunk-validate
             count = 2
         mean rate = 0.96 calls/second
     1-minute rate = 0.00 calls/second
     5-minute rate = 0.00 calls/second
    15-minute rate = 0.00 calls/second
               min = 5.74 milliseconds
               max = 15.99 milliseconds
              mean = 10.87 milliseconds
            stddev = 5.13 milliseconds
            median = 15.99 milliseconds
              75% <= 15.99 milliseconds
              95% <= 15.99 milliseconds
              98% <= 15.99 milliseconds
              99% <= 15.99 milliseconds
            99.9% <= 15.99 milliseconds


Total execution time (sec): 4
Failures: 1
Successful executions: 1

Issue Links

• links to: GitHub Pull Request #2850