Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: consensus, log
    • Target Version/s:

      Description

      WAL log can take significent disk space. So far there are some config to limit it. but it can go very high.
      WAL log size = #tablets * log_segment_size_mb * log segments (1 if there is write ops to this tablet, can go up to log_max_segments_to_retain)
      Logs are retained even if there is no write for a while.

      We could reduce the WAL log usage by

      • reduce min_segments_to_retain to 1 instead of 2, a
      • reduce steady state consumption of idle tablets, roll a WAL if it has had no writes for a few minutes and size more than a MB or two so that "idle" tablets have 0 WAL space consumed

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                jyu@cloudera.com Juan Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: