[CALCITE-5730] Initial null values can be dropped by EnumerableLimitSort with offset

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.35.0
• Component/s: linq4j
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Feng Guo
• Created: 29/May/23 08:14
• Updated: 27/Jul/23 10:22
• Resolved: 02/Jun/23 06:32

Description

Description

The EnumerableSortLimit Node deals with offset with following logic:

// skip the first 'offset' rows by deleting them from the map
if (offset > 0) {
  // search the key up to (but excluding) which we have to remove entries from the map
  int skipped = 0;
  TKey until = null;
  for (Map.Entry<TKey, List<TSource>> e : map.entrySet()) {
    skipped += e.getValue().size();

    if (skipped > offset) {
      // we might need to remove entries from the list
      List<TSource> l = e.getValue();
      int toKeep = skipped - offset;
      if (toKeep < l.size()) {
        l.subList(0, l.size() - toKeep).clear();
      }

      until = e.getKey();
      break;
    }
  }
  if (until == null) {
    // the offset is bigger than the number of rows in the map
    return Linq4j.emptyEnumerator();
  }
  map.headMap(until, false).clear();
}

In a NULLS FIRST sort, if we set offset=1, limit = 1 when first 10 rows have null compare key, the until will be null. But that does not mean offset bigger than number of rows, it should have results instead of emptyEnumerator;

Issue Links

• links to: GitHub Pull Request #3228