Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-2494

Fix usage of different key generators with metadata enabled

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      With sync metadata patch, when metadata is enabled by default, some spark datasource tests failed which were using timestamp based key gen and custom key gen. Metadata table's records are getting picked up when we do 

       

      spark.read.format(hudi).load(basePath + "/*/*")
      

       

      For now, I have disabled metadata for these tests. 

      testSparkPartitonByWithTimestampBasedKeyGenerator

      testSparkPartitonByWithCustomKeyGenerator

       

      I was looking at options to ignore certain path, but looks like there is none. 

       

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shivnarayan sivabalan narayanan
            shivnarayan sivabalan narayanan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment