Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: jackrabbit-core
    • Labels:
      None

      Description

      Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.

      Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.

      What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.

        Issue Links

          Activity

          Hide
          Michael Dürig added a comment -

          I encountered this issue while investigating the root cause for JCR-3473. The problem already occurs on running the query (no need to fetch the result rows/nodes). I wonder what the original expectation for queries was. The presence of LargeResultSetTest leads me to believe that such queries should actually work but unfortunately JCR-3472 prevented us from detecting they don't. If OTOH we don't except these queries to work, we need to find a way to fix JCR-3473 without having to rely on queries.

          Show
          Michael Dürig added a comment - I encountered this issue while investigating the root cause for JCR-3473 . The problem already occurs on running the query (no need to fetch the result rows/nodes). I wonder what the original expectation for queries was. The presence of LargeResultSetTest leads me to believe that such queries should actually work but unfortunately JCR-3472 prevented us from detecting they don't. If OTOH we don't except these queries to work, we need to find a way to fix JCR-3473 without having to rely on queries.
          Hide
          Marcel Reutegger added a comment -

          At least my expectation is that Jackrabbit should be able to handle large result sets with reasonable memory usage. Do we have more details on what exactly consumes that much memory? Is it specific to some queries or is it for any kind of query?

          Show
          Marcel Reutegger added a comment - At least my expectation is that Jackrabbit should be able to handle large result sets with reasonable memory usage. Do we have more details on what exactly consumes that much memory? Is it specific to some queries or is it for any kind of query?
          Hide
          Michael Dürig added a comment - - edited

          I have so far seen it for queries of type

          /jcr:root/root//element

          when the tree rooted at "root" is large (no flat hierarchies involved). I.e. ~110k nodes bumps memory usage up to 400MB on Query.execute().

          Show
          Michael Dürig added a comment - - edited I have so far seen it for queries of type /jcr:root/root//element when the tree rooted at "root" is large (no flat hierarchies involved). I.e. ~110k nodes bumps memory usage up to 400MB on Query.execute().
          Hide
          Marcel Reutegger added a comment -

          IIRC, these kind of queries are special cased because it basically means traversing the hierarchy. Maybe something is wrong there...

          Show
          Marcel Reutegger added a comment - IIRC, these kind of queries are special cased because it basically means traversing the hierarchy. Maybe something is wrong there...

            People

            • Assignee:
              Unassigned
              Reporter:
              Alex Parvulescu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development