Description
InferFiltersFromConstraints optimization rule generates IsNotNull constraints corresponding to null intolerant predicates. The IsNotNull constraints are generated on the attribute inside the corresponding predicate.
e.g. A predicate a > 0 on an integer column a will result in a constraint IsNotNull(a). On the other hand a predicate on a nested int column structCol.b where structCol is a struct column results in a constraint IsNotNull(structCol).
This generation of constraints on the root level nested type is extremely conservative as it could lead to materialization of the the entire struct. The constraint should instead be generated on the nested field being referenced by the predicate. In the above example, the constraint should be IsNotNull(structCol.b) instead of IsNotNull(structCol)
The new constraints also create opportunities for nested pruning. Currently IsNotNull(structCol) constraint would preclude pruning of structCol. However the constraint IsNotNull(structCol.b) could create opportunities to prune structCol.