Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4155

Support optional Source schema config for S3EventsHoodieIncrSource

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      `S3EventsHoodieIncrSource`  does not support external optional schema but infers schema using spark . But there are use cases for optional fields to be added in dataframe reader to support transformers which are dependent on optional fields . 
      Also not necessary infer schema fits every requirement . 

      Attachments

        Issue Links

          Blocked

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. HUDI-1896 HudiStreamer Source for cloud object stores

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            People

              harsh1231 Harshal Patil
              harsh1231 Harshal Patil
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: