Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15226 Über-JIRA: S3Guard Phase III: Hadoop 3.2 features
  3. HADOOP-15480

AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.2.0
    • fs/s3
    • None

    Description

      When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the testDiffCommand test fails with the following:

      testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)  Time elapsed: 8.059 s  <<< FAILURE!
      java.lang.AssertionError: 
      Mismatched metadata store outputs: MS	D	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
      MS	F	100	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
      MS	F	100	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
      MS	F	100	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
      MS	F	100	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
      MS	F	100	s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
      S3	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
      MS	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
      S3	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
      MS	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
      S3	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
      MS	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
      S3	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
      MS	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
      S3	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
      MS	F	0	s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
       expected:<[
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]> 
      
      but was:<[
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4, 
      s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      

      Attachments

        1. HADOOP-15480.005.patch
          5 kB
          Aaron Fabbri
        2. HADOOP-15480.004.patch
          5 kB
          Aaron Fabbri
        3. HADOOP-15480.003.patch
          5 kB
          Aaron Fabbri
        4. HADOOP-15480.002.patch
          5 kB
          Gabor Bota
        5. HADOOP-15480.001.patch
          3 kB
          Gabor Bota

        Activity

          People

            gabor.bota Gabor Bota
            gabor.bota Gabor Bota
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: