Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-5335

Make metasync to multiple catalogs multithreaded

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • meta-sync
    • None

    Description

      If you have multiple catalogs, metasync to each one happens sequentially. If a user needs this to be multithreaded it can be implemented pretty easily. https://github.com/apache/hudi/pull/7318 is an example of how to do it

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HUDI-5295 With multiple meta syncs, one meta sync failure should not impact other meta syncs.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              jonvex Jonathan Vexler
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: