XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Testing
    • Labels:
      None

      Description

      We plan to enable metadata by default. but there are some tests that fail with this. Dumping details on tests for which metadata is disabled for now. We need to fix them one by one.  

       

      hudi-spark-client: // this is the module that has lot of tests that could potentially have issues. 

      TestHoodieSparkMergeOnReadTableIncrementalRead.testIncrementalReadsWithCompaction. disabled metadata for now. directly accesses files.

      TestHoodieIndex.
      testSimpleTagLocationAndUpdateWithRollback. known issue.  https://issues.apache.org/jira/browse/HUDI-2468

      testSimpleGlobalIndexTagLocationWhenShouldUpdatePartitionPath. uses test table. disabled metadata.

      TestHoodieRowCreateHandle.testInstantiationFailure. disabled metadata. not a real issue. 

      TestHoodieCompactor.testWriteStatusContentsAfterCompaction. uses HoodieSparkWriteableTestTable. have disabled metadata.
      TestHoodieSparkMergeOnReadTableRollback.testMultiRollbackWithDeltaAndCompactionCommit. restore fails. bcoz, there is an inflight rollback in dataset timeline. disabling for now. https://issues.apache.org/jira/browse/HUDI-2477
      TestHoodieMergeOnReadTable.testLogFileCountsAfterCompaction. uses HoodieSparkWriteableTestTable. disabled metadata for now.
      TestHoodieCompactor.testWriteStatusContentsAfterCompaction. uses HoodieSparkWriteableTestTable. have disabled metadata.
      TestHbaseIndex.testEnsureTagLocationUsesCommitTimeline. rolling back 1st commit. known issue. disabling metadata. https://issues.apache.org/jira/browse/HUDI-2468
      TestHbaseIndex.testSimpleTagLocationAndUpdateWithRollback. rolling back 1st commit. known issue. disabling metadata. https://issues.apache.org/jira/browse/HUDI-2468

      TestCleaner. lot of tests. uses test table.
      TestHoodieTimelineArchiveLog. lot of tests. uses test table. 

      TestHoodieSparkMergeOnReadTableRollback.testRollbackWithDeltaAndCompactionCommit. uses test table.

      TestMergeOnReadRollbackActionExecutor.testRollbackWhenFirstCommitFail known issue. https://issues.apache.org/jira/browse/HUDI-2468

      hudi-utilities:

      TestHoodieDeltaStreamer. testCleanerDeleteReplacedDataWithArchive. fails. relating to archival. disabling metadata. need to look into it.

      hudi-client-common: all passed.
      hudi-flink-client: all passed.
      hudi-java-client: disabled metadata for java. all ok.
      hudi-common: all passed.
      hudi-spark java: Testbootstrap class fully fails. rollback of 1st commit. have disbaled metadata. https://issues.apache.org/jira/browse/HUDI-2477

      hudi-spark scala tests: all good except couple of key gen related failures (https://issues.apache.org/jira/browse/HUDI-2494)

      hudi-utilities: one test in deltastreamer.
      hudi-timelineserver: all good.
      hudi-sync:
          hudi-dla-sync: all good.
          hudi-hive-sync: all good.
      hudi-spark3: all good.
      hudi-spark2: all good.
      hudi-examples: no tests.

       

       

       

      Disabled flink tests:

      TestStreamWriteOperatorCoordinator.testSyncMetadataTable
      TestFileIndex.testFileListingEmptyTable

       

       

       

       

       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                manojg Manoj Govindassamy
                Reporter:
                shivnarayan sivabalan narayanan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: