Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1487

after HUDI-1376 merged unit test testCopyOnWriteStorage will failed random

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None

      Description

       

       

      TestCOWDataSource.testCopyOnWriteStorage will failed random. Because  before the  incremental read, add a new upsert commit.

      // pull the latest commit
      val hoodieIncViewDF2 = spark.read.format("org.apache.hudi")
      .option(DataSourceReadOptions.QUERY_TYPE_OPT_KEY, DataSourceReadOptions.QUERY_TYPE_INCREMENTAL_OPT_VAL)
      .option(DataSourceReadOptions.BEGIN_INSTANTTIME_OPT_KEY, commitInstantTime2)
      .load(basePath)

      the new commit is :

      // Upsert based on the written table with Hudi metadata columns
      val verificationRowKey = snapshotDF1.limit(1).select("_row_key").first.getString(0)

      as verificationRowKey will contains in "uniqueKeyCnt", so will failed as : "expected: <65> but was: <66>"

       

       

      https://travis-ci.com/github/apache/hudi/jobs/463879606

       

      org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:173)
      at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlices(RemoteHoodieTableFileSystemView.java:275)
      ... 30 more
      [WARN ] 2020-12-22 12:32:40,788 org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
      [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 35.352 s <<< FAILURE! - in org.apache.hudi.functional.TestCOWDataSource
      [ERROR] org.apache.hudi.functional.TestCOWDataSource.testCopyOnWriteStorage Time elapsed: 15.275 s <<< FAILURE!
      org.opentest4j.AssertionFailedError: expected: <65> but was: <66>
      at org.apache.hudi.functional.TestCOWDataSource.testCopyOnWriteStorage(TestCOWDataSource.scala:160)
      [INFO] Running org.apache.hudi.functional.TestDataSourceForBootstrap
      [WARN ] 2020-12-22 12:32:43,641 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:32:47,818 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:32:50,921 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:32:56,169 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:32:56,793 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:32:57,388 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:33:05,191 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:33:10,221 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:33:17,985 org.apache.hudi.DefaultSource - Loading Base File Only View.
      [WARN ] 2020-12-22 12:33:22,498 org.apache.hudi.DefaultSource - Loading Base File Only View.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                309637554 liwei
                Reporter:
                309637554 liwei
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: