Uploaded image for project: 'Sentry (Retired)'
  1. Sentry (Retired)
  2. SENTRY-1912

Directly construct Thrift message from DB when full update is sent to NN

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.0
    • None
    • Sentry
    • None

    Description

      Currently when we send full update to the NameNode we do the following:

      1. Read full snapshot from DB into Map<String, Set<String>> structure
      2. Convert it to Thrift structure which uses list of strings.

      We should be able to directly construct Thrift version without going to the intermediate representation. This should save significant amount of memory since we don't have to maintain two copies of the same data in memory and we don't need to use inefficient Set data structures for huge objects.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SENTRY-1909 Improvements for memory usage when full path snapshot is sent from Sentry to NN

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              akolb Alex Kolbasov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: