Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Noticed this when testing the patch for HADOOP-13756.

       

      [ERROR] Failures:
      
      [ERROR]   ITestS3GuardToolLocal>AbstractS3GuardToolTestBase.testPruneCommandCLI:221->AbstractS3GuardToolTestBase.testPruneCommand:201->AbstractS3GuardToolTestBase.assertMetastoreListingCount:214->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 Pruned children count [PathMetadata{fileStatus=S3AFileStatus{path=s3a://bucket-new/test/testPruneCommandCLI/stale; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1524798258286; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}, PathMetadata{fileStatus=S3AFileStatus{path=s3a://bucket-new/test/testPruneCommandCLI/fresh; isDirectory=false; length=100; replication=1; blocksize=512; modification_time=1524798262583; access_time=0; owner=hdfs; group=hdfs; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} isEmptyDirectory=FALSE; isEmptyDirectory=UNKNOWN; isDeleted=false}] expected:<1> but was:<2>

       

      Looking through the code, I'm noticing a couple of issues.

       

      1. testDiffCommand() is in ITestS3GuardToolLocal, but it should really be running for all MetadataStore implementations.  Seems like it should live in AbstractS3GuardToolTestBase.

      2. AbstractS3GuardToolTestBase#createFile() seems wrong. When onMetadataStore is false, it does a ContractTestUtils.touch(file), but the fs is initialized with a MetadataStore present, so seem like the fs will still put the file in the MetadataStore?

      There are other tests which explicitly go around the MetadataStore by using fs.setMetadataStore(nullMS), e.g. ITestS3AInconsistency. We should do something similar in AbstractS3GuardToolTestBase#createFile(), minding any issues with parallel test runs.

        Attachments

        1. HADOOP-15420.001.patch
          9 kB
          Gabor Bota
        2. HADOOP-15420.002.patch
          9 kB
          Gabor Bota

          Issue Links

            Activity

              People

              • Assignee:
                gabor.bota Gabor Bota
                Reporter:
                fabbri Aaron Fabbri
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: