Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3199

Hive sync config unification

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.12.0
    • configs, meta-sync
    • None

    Description

      There are some hive sync config in flink, spark api, deltastreamer and kafka-connect.

      But these properties are in hudi-spark-common package, so that some services like flink or kafka-connect can not use it directly (related classes use scala or spark references).
      At present, a problem caused by class reference can be referred to: https://issues.apache.org/jira/browse/HUDI-3112

      We should unify hive sync config (like move to hudi-hive-sync), so that it can be synchronized to other packages when it is updated.

      Attachments

        Issue Links

          is fixed by

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. HUDI-3730 RFC-55: Improve hudi-sync classes design and simplify configs

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Task - A task that needs to be done. HUDI-3112 KafkaConnect can not sync to Hive

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              xushiyan Shiyan Xu
              chenxiang cdmikechen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: