Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3649

Separate compression setting for flush files

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In this thread on user@hbase: http://search-hadoop.com/m/WUnLM6ojHm1 J-D conjectures that compressing flush files leads to a suboptimal situation where "the puts are sometimes blocked on the memstores which are blocked by the flusher thread which is blocked because there's too many files to compact because the compactor is given too many small files to compact and has to compact the same data a bunch of times."

      We have a separate compression setting already for major compaction vs store files written during minor compaction, for background/archival apps. Add a separate compression setting for flush files, default to none, to avoid the above condition.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              apurtell Andrew Purtell
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: