Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-1912

Directly construct Thrift message from DB when full update is sent to NN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: Sentry
    • Labels:
      None

      Description

      Currently when we send full update to the NameNode we do the following:

      1. Read full snapshot from DB into Map<String, Set<String>> structure
      2. Convert it to Thrift structure which uses list of strings.

      We should be able to directly construct Thrift version without going to the intermediate representation. This should save significant amount of memory since we don't have to maintain two copies of the same data in memory and we don't need to use inefficient Set data structures for huge objects.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                akolb Alex Kolbasov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: