This issue originates from
SPARK-11103, which contains detailed information about how to reproduce it.
The major problem here is that, filter predicates pushed down assert that columns they touch must exist in the target physical files. But this isn't true in case of schema merging.
Actually this assertion is unnecessary, because if a column is missing in the filter schema, the column is considered to be filled by nulls, and all the filters should be able to act accordingly. For example, if we push down a = 1 but a is missing in the underlying physical file, all records in this file should be dropped since a is always null. On the other hand, if we push down a IS NULL, all records should be preserved.