Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2238

Big DMS not flush under memory pressure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.7.0
    • None
    • None
    • CentOS6.5 Linux 2.6.32-431
      Kudu1.3.0
      GitCommit 00813f96b9cb

    Description

      I have a table with many updates, its DMS consumes a lot memory and cause “Soft Memory Limit Exceed”. I check the /mem-trackers on the tablet server, one of its DMS consumes about 3G memory, but check the /maintenance-manager, its FlushDeltaMemStoresOp can only free 763B anchored memory and perf_improvement is 0. Is this normal? I know Kudu is not optimized for updates, but still confused why the DMS won’t be flushed under memory pressure.

      Infos from /mem-trackers:

      tablet-5941a8bb934e4686abd1bfff9e35c860 server none 3.00G 3.00G
      txn_tracker tablet-5941a8bb934e4686abd1bfff9e35c860 64.00M 0B 1.67M
      MemRowSet-339 tablet-5941a8bb934e4686abd1bfff9e35c860 none 265B 265B
      DeltaMemStores tablet-5941a8bb934e4686abd1bfff9e35c860 none 3.00G 3.00G

      Infos from /maintenance-manager

      FlushDeltaMemStoresOp(5941a8bb934e4686abd1bfff9e35c860) true 763B 511.15M 0

      The tablet 5941a8bb934e4686abd1bfff9e35c860 has 16 RowSets in total

      Some configs of MM:
      --enable_maintenance_manager=true
      --log_target_replay_size_mb=1024
      --maintenance_manager_history_size=8
      --maintenance_manager_num_threads=6
      --maintenance_manager_polling_interval_ms=50

      Attachments

        1. memory_anchored.png
          31 kB
          ZhangZhen
        2. memory_consumed.png
          96 kB
          ZhangZhen

        Activity

          People

            zhquake@gmail.com ZhangZhen
            zhquake@gmail.com ZhangZhen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: