I have some code that I’m running in a unit test suite, but the code I’m running is failing with an assertion error.
I have translated the JUnit test that was failing, to a Scala script that I will attach to the ticket. The assertion error is the following:
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.AssertionError: lengthInBytes must be a multiple of 8 (word-aligned)
However, it turns out that this code actually works normally and computes the correct result if assertions are turned off.
I traced the code and found that when hashUnsafeWords was called, it was given a byte-length of 12, which clearly is not a multiple of 8. However, the job seems to compute correctly regardless of this fact. Of course, I can’t just disable assertions for my unit test though.
A few things we need to understand:
1. Why is the lengthInBytes of size 12?
2. Is it actually a problem that the byte length is not word-aligned? If so, how should we fix the byte length? If it's not a problem, why is the assertion flagging a false negative?