If I/O encryption is enabled in Spark (
SPARK-5682), the write ahead logs written by streaming applications will also be encrypted.
That creates issues because only the driver that wrote those files knows the key to decrypt them. So basically the WAL is unusuable in that scenario.
Instead, the WAL should not be encrypted by Spark; if the user has a need to encrypt that data, other means (such as HDFS encryption zones) should be used instead.