Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2278

Improve IO for writing deltas

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: cfile, perf, tablet
    • Labels:
      None

      Description

      Today, writing new deltas entails rewriting entire tablet metadata files in order to track the newly-created block ids. Even if the delta were on the order of kilobytes, the tablet metadata files could be on the order of megabytes, so the relative cost for this small amount of data is quite high, considering the amount of the metadata.

      This could be improved by batching such delta flushes, or by revamping tablet metadata entirely to batch any operations that require metadata updates.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                awong Andrew Wong
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: