The problem occurs with the simple code below:
It takes several minutes and a very high Java heap usage, when it should be immediate.
It does not occur when replacing the unique integer value (1) with a string value ("x").
All the time is spent in the PruneFilters optimization rule.
Not reproduced in Spark 2.4.1.