Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-507

Support \ t split hdfs source

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Utilities
    • Labels:
      None

      Description

      hi,hudi

       

      Current Hudi data source does not support HDFS file data splitting with \ t separator
      I want to complete it and contribute to the community.
      The main change is the addition of the TextDFSSource class to provide support.
      The specific new logic is: split the hdfs data according to the delimiter, and then map it to the source.avsc pattern

       

      Or do some other symbol format as an extension

      thanks,

      liujh

       

      Vinoth Chandar   Please help with suggestions

       

        Attachments

          Activity

            People

            • Assignee:
              liujinhui liujinhui
              Reporter:
              liujinhui liujinhui
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 240h
                240h
                Remaining:
                Remaining Estimate - 240h
                240h
                Logged:
                Time Spent - Not Specified
                Not Specified