Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-1403

Perf and stability improvements to DSL search and lineage to search only by typeNames

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7-incubating
    • Component/s: None
    • Labels:
      None

      Description

      Currently DSL uses a fill function during Gremlin Translation to merge results by typeName and superTypeName and fill function loads the resulting vertices in memory. This causes significant memory usage and ATLAS server spends lot of time doing GC instead of useful work resulting in OOO sometimes ( when GC is not able to recover and search queries are run in parallel)

      The proposal is to replace this with typeName checks along by finding all the subtypes for a given type and using an IN clause in the filter.

      For eg:

      Query = Person where (birthday < "1950-01-01T02:35:58.440Z") limit 40 offset 0

      Optimized query

      Gremlin Query = L:

      {g.V.has("__typeName", T.in, ['Person','Manager']).and(_().has("Person.birthday", T.lt, -631142641560)) [0..<40].toList()}

        Attachments

        1. ATLAS-1403_0.7-incubating.patch
          40 kB
          Madhan Neethiraj
        2. ATLAS-1403.1.patch
          28 kB
          Sarath Subramanian
        3. ATLAS-1403.6.patch
          29 kB
          Sarath Subramanian

          Activity

            People

            • Assignee:
              suma.shivaprasad Suma Shivaprasad
              Reporter:
              suma.shivaprasad Suma Shivaprasad

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment