Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7667 Support stripe compaction
  3. HBASE-8541

implement flush-into-stripes in stripe compactions

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.98.0, 0.99.0
    • None
    • None

    Description

      Flush will be able to flush into multiple files under this design, avoiding L0 I/O amplification.
      I have the patch which is missing just one feature - support for concurrent flushes and stripe changes. This can be done via extensive try-locking of stripe changes and flushes, or advisory flags without blocking flushes, dumping conflicting flushes into L0 in case of (very rare) collisions. For file loading for the latter, a set-cover-like problem needs to be solved to determine optimal stripes. That will also address Jimmy's concern of getting rid of metadata, btw. However currently I don't have time for that. I plan to attach the try-locking patch first, but this won't happen for a couple weeks probably and should not block main reviews. Hopefully this will be added on top of main reviews.

      Attachments

        1. HBASE-8541-latest-with-dependencies.patch
          252 kB
          Sergey Shelukhin
        2. HBASE-8541-latest-with-dependencies.patch
          248 kB
          Sergey Shelukhin
        3. HBASE-8541-latest-with-dependencies.patch
          336 kB
          Sergey Shelukhin
        4. HBASE-8541-latest-with-dependencies.patch
          336 kB
          Sergey Shelukhin
        5. HBASE-8541-v0.patch
          61 kB
          Sergey Shelukhin
        6. HBASE-8541-v1.patch
          62 kB
          Sergey Shelukhin
        7. HBASE-8541-v2.patch
          50 kB
          Sergey Shelukhin
        8. HBASE-8541-v3.patch
          54 kB
          Sergey Shelukhin
        9. HBASE-8541-v4.patch
          53 kB
          Sergey Shelukhin
        10. HBASE-8541-v5.patch
          54 kB
          Sergey Shelukhin

        Activity

          People

            sershe Sergey Shelukhin
            sershe Sergey Shelukhin
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: