Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-617

Add support for data types convertible to String in TimestampBasedKeyGenerator

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Currently, TimestampBasedKeyGenerator only supports 4 data types for the partition key. They areĀ  Double, Long, Float and String. However, if the `avro.java.string` is not specified in the schema provided, Hudi throws the following error:

      org.apache.hudi.exception.HoodieNotSupportedException: Unexpected type for partition field: org.apache.avro.util.Utf8
      at org.apache.hudi.utilities.keygen.TimestampBasedKeyGenerator.getKey(TimestampBasedKeyGenerator.java:111)
      at org.apache.hudi.utilities.deltastreamer.DeltaSync.lambda$readFromSource$f92c188c$1(DeltaSync.java:338)
      ....

      It will be better if the support was more generalised to include the data types that provide method to convert them to String such as `Utf8` since all these methods implement the `CharSequence` interface.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            amitds1997 Amit Singh
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Issue deployment