Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.20.2
-
None
Description
When analyzing data, there is currently no single function to evaluate whether a given field is empty. With scalar fields, this can be accomplished with the `IS NOT NULL` operator, but with complex fields, this is more challenging as complex fields are never null.
This PR adds a UDF called IS_EMPTY() which accepts any type of field and returns true if the field does not contain data.
In the case of scalar fields, if the field is `null` this returns true. In the case of complex fields, which can never be `null`, in the case of lists, the function returns true if the list is empty. In the case of maps, it returns true if all of the map's fields are unpopulated.