Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4038

Avoid invoking `getDataSize` in the hot-path

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.11.1
    • None

    Description

      `getDataSize` has non-trivial overhead of traversing already encoded Column Groups stored in memory. We should sample its invocations to amortize its costs.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HUDI-3995 Bulk insert row writer perf improvements

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          links to

          Web Link GitHub Pull Request #5497

          Activity

            People

              alexey.kudinkin Alexey Kudinkin
              alexey.kudinkin Alexey Kudinkin
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: