Major Description
In KuduRestore.scala we have code like
// Fail the task if there are any errors.
val errorCount = session.getPendingErrors.getRowErrors.length
if (errorCount > 0) {
val errors =
session.getPendingErrors.getRowErrors.take(5).map(_.getErrorStatus).mkString
throw new RuntimeException(
s"failed to write $errorCount rows from DataFrame to Kudu; sample errors: $errors")
}
There's similar code in KuduContext.scala:
val errorCount = pendingErrors.getRowErrors.length
if (errorCount > 0) {
val errors =
pendingErrors.getRowErrors.take(5).map(_.getErrorStatus).mkString
throw new RuntimeException(
s"failed to write $errorCount rows from DataFrame to Kudu; sample errors: $errors")
}
I've seen the former fail to print any sample errors. Taking a reference to session.getPendingErrors.getRowErrors and using that through fixes this, so it seems like there's some TOCTOU problem that can occur, probably because multiple batches can be in flight at once.
The latter is most likely vulnerable to this as well.
This issue made diagnosing KUDU-2809 harder.