Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-1915

Sentry is doing a lot of work to convert list of paths to HMSPaths structure

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: Sentry
    • Labels:
      None

      Description

      It turns out that in 2.0 we changed the way full snapshots are sent from Sentry to HDFS. Before they were using HMSPaths which used tree structure and eliminated some duplication. Also SENTRY-1827 helped to compressed this on the serialization side.

      Now we are using TPathChanges structure that is not tree-based and contains very non-efficient way of representing paths: required list<list<string>> addPaths; so we split each paths on slashes and store list of elements instead of storing a tree. As a result we may use much more memory.

        Attachments

        1. SENTRY-1915.01.patch
          12 kB
          Alex Kolbasov
        2. SENTRY-1915.02.patch
          18 kB
          Alex Kolbasov

          Issue Links

            Activity

              People

              • Assignee:
                akolb Alex Kolbasov
                Reporter:
                akolb Alex Kolbasov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: