Details

    • Type: Sub-task
    • Status: Reopened
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.8.0
    • Fix Version/s: None
    • Component/s: fs/s3, test
    • Labels:
      None

      Description

      I'm seeing an intermittent failure of TestS3AContractRootDir.testListEmptyRootDirectory

      The sequence of : deleteFiles(listStatus(Path("/)")) is failing because the file to delete is root ...yet the code is passing in the children of /, not / itself.

      hypothesis: when you call listStatus on an empty root dir, you get a file entry back that says isFile, not isDirectory.

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          seeing this now sometimes as bad auth, which means it'll need the retry logic of HADOOP-14531 to make progress. Marking as depedendent & moving out of phase III

          Show
          stevel@apache.org Steve Loughran added a comment - seeing this now sometimes as bad auth, which means it'll need the retry logic of HADOOP-14531 to make progress. Marking as depedendent & moving out of phase III
          Hide
          stevel@apache.org Steve Loughran added a comment -

          HADOOP-14851 is probably the reason we haven't fixed this: even though there's a loop, because eventually() isn't looping around intercepts, there's no iteration when there's a (transient) list inconsistency

          Show
          stevel@apache.org Steve Loughran added a comment - HADOOP-14851 is probably the reason we haven't fixed this: even though there's a loop, because eventually() isn't looping around intercepts, there's no iteration when there's a (transient) list inconsistency
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This has stopped for me recently, not sure what is up. I think whatever consistency condition existed, it has "gone away" with changes to the code.

          maybe close as cannot-reproduce until it comes back

          Show
          stevel@apache.org Steve Loughran added a comment - This has stopped for me recently, not sure what is up. I think whatever consistency condition existed, it has "gone away" with changes to the code. maybe close as cannot-reproduce until it comes back
          Hide
          stevel@apache.org Steve Loughran added a comment -
          java.io.IOException: Root directory operation rejected: s3a://steve-ireland/
          	at org.apache.hadoop.fs.contract.ContractTestUtils.rejectRootOperation(ContractTestUtils.java:384)
          	at org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:569)
          	at org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:552)
          	at org.apache.hadoop.fs.contract.AbstractContractRootDirectoryTest.testListEmptyRootDirectory(AbstractContractRootDirectoryTest.java:132)
          	at org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir.testListEmptyRootDirectory(TestS3AContractRootDir.java:49)
          

          Note this is happening with parallel execution, but in the sequential bit. Rerunning the test suiite standalone works.

          Show
          stevel@apache.org Steve Loughran added a comment - java.io.IOException: Root directory operation rejected: s3a: //steve-ireland/ at org.apache.hadoop.fs.contract.ContractTestUtils.rejectRootOperation(ContractTestUtils.java:384) at org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:569) at org.apache.hadoop.fs.contract.ContractTestUtils.assertDeleted(ContractTestUtils.java:552) at org.apache.hadoop.fs.contract.AbstractContractRootDirectoryTest.testListEmptyRootDirectory(AbstractContractRootDirectoryTest.java:132) at org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir.testListEmptyRootDirectory(TestS3AContractRootDir.java:49) Note this is happening with parallel execution, but in the sequential bit. Rerunning the test suiite standalone works.

            People

            • Assignee:
              Unassigned
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development