[ASTERIXDB-3410] OOM Error with Heavily Nested Queries

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: COMP - Compiler
• Labels: triaged
• Assignee: Unassigned
• Reporter: Glenn Justo Galvizo
• Created: 29/May/24 22:24
• Updated: 23/Aug/24 17:48
• Resolved: 23/Aug/24 17:48

Description

Something I came across while trying to use AFrame with AsterixDB (https://dzone.com/articles/dataframes-with-big-data-in-couchbase). A pattern used by AFrame and early versions of Graphix involved repeatedly nesting queries:

SELECT VALUE OBJECT_REMOVE(t, 'categories') 
FROM (
  SELECT t.*, to_bigint(contains(categories, "Home Services")) AS `Home Services` 
  FROM (
    SELECT t.*, to_bigint(contains(categories, "Bars")) AS `Bars`
    FROM ...
    
    .
    .
    .
    
  ) t
) t

 

Unfortunately, for heavily nested queries, AsterixDB throws an out of memory error. This was an issue I faced with the compiler, not the runtime (I ran this with EXPLAIN, and I still got the OOM error).

 

Potential remedy: we do have a decorrelation Algebricks rule, but it may (potentially) be less memory intensive to have an AST-level rule that tries to do the same.