Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19092 ABFS phase 4: post Hadoop 3.4.0 features
  3. HADOOP-18960

ABFS contract-tests with Hadoop-Commons intermittently failing

    XMLWordPrintableJSON

Details

    Description

      In the merged pr HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path by anujmodi2021 · Pull Request #6003 · apache/hadoop (github.com), a config was switched-on: `fs.contract.test.root-tests-enabled`. This enables the root manipulation tests for the filesystem contract.

      Now, the execution of contract-tests in abfs works as per executionId integration-test-abfs-parallel-classes of the pom. The tests would work in different jvms, and at a given instance multiple such jvms could be there, depending on ${testsThreadCount}.  The problem is that all the test jvms for contract-test use the same container for test runs which is defined by `fs.contract.test.fs.abfs`. Due to this, one jvm root-contract-runs can influence other jvm's root-contract-runs. This leads to CI failures for hadoop-azure package.

      Solution is to run these tests sequentially and separate from other commit/manifest tests.

      Attachments

        Activity

          People

            anujmodi Anuj Modi
            pranavsaxena Pranav Saxena
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: