Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3056

Not able to query hudi tables in 0.10.0 with redshift

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 0.10.0
    • None
    • Usability

    Description

      Since with 0.10.0 we changed the format of timestamp, redshift is failing to read hudi tables created with 0.10.0 

       

      I have noticed the instant timestamp length have changed from yyyyMMddHHmmss to yyyyMMddHHmmssSSS,
      once I tried to use 'org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat', redshift shows 'hudi::ParsedFilename::IsValidCommitTimestamp( std::string(ctx.hudi_commit_timestamp))' error, I still have no idea why there is nothing return when using org.apache.hudi.hadoop.HoodieParquetInputFormat

      {{}}

      Ref issue: https://github.com/apache/hudi/issues/4283

      {{}}

      {{}}

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            uditme Udit Mehrotra Assign to me
            shivnarayan sivabalan narayanan

            Dates

              Created:
              Updated:

              Agile

                Completed Sprints:
                Cont' improve - 2022/02/14 ended 22/Feb/22
                Cont' improve - 2022/02/21 ended 01/Mar/22
                View on Board

                Slack

                  Issue deployment