Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1967 Umbrella JIRA for node density improvements
  3. KUDU-1971

Explore reducing number of data blocks by tuning existing parameters

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4.0
    • Fix Version/s: None
    • Component/s: tablet

      Description

      One way to scale to larger on-disk data sets is to reduce the ratio between data blocks and data; that is, to make data blocks larger. Two existing parameters control for this:

      • budgeted_compaction_target_rowset_size: within a given flush or compaction operation, stipulates the size of each rowset. Currently 32M.
      • tablet_compaction_budget_mb: stipulates the amount of data that should be included in any given compaction. Currently 128M.

      It might be interesting to explore raising these.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              adar Adar Dembo
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: