Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-3197

Tablet keeps all history schemas in memory may result in high memory consumption

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.12.0
    • Fix Version/s: None
    • Component/s: tablet
    • Labels:
      None

      Description

      In case of high frequency of updating table, memory consumption of kudu-tserver may be very high, and the memory in not tracked in the memory page. 

      This is the memory usage of a tablet, the memory consumption of tablet-xxx‘s peak is 3.6G, but none of its' childrens' memory can reach.

      So I use pprof to get the heap sampling. The tserver started for long but the memory is still consuming by TabletBootstrap:PlayAlterSchemaRequest. 

      I change the `old_schemas_` in tablet_metadata.h to a fixed size vector,

        // Previous values of 'schema_'.
      // These are currently kept alive forever, under the assumption that
      // a given tablet won't have thousands of "alter table" calls.
      // They are kept alive so that callers of schema() don't need to
      // worry about reference counting or locking.
      std::vector<Schema*> old_schemas_;

      The heap sampling then becomes

      So, to make application layer more flexible, it could be better to make the size of the old_schemas configurable.
       

        Attachments

        1. screenshot-1.png
          162 kB
          wangningito
        2. image-2020-12-02-19-59-46-733.png
          162 kB
          wangningito
        3. image-2020-09-25-15-05-44-948.png
          97 kB
          wangningito
        4. image-2020-09-25-14-49-30-913.png
          162 kB
          wangningito
        5. image-2020-09-25-14-45-33-402.png
          229 kB
          wangningito

          Activity

            People

            • Assignee:
              wangning wangningito
              Reporter:
              wangning wangningito
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: