Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.1, 2.0.3-alpha
    • Fix Version/s: 3.0.0
    • Component/s: test
    • Labels:
      None

      Description

      The File System Contract contains implicit assumptions that aren't checked in the contract test base. Add more tests to define the contract's assumptions more rigorously for those filesystems that are tested by this (not Local, BTW)

      1. HADOOP-9258-8.patch
        28 kB
        Steve Loughran
      2. HADOOP-9528.patch
        5 kB
        Steve Loughran
      3. HADOOP-9528-2.patch
        6 kB
        Steve Loughran
      4. HADOOP-9528-3.patch
        9 kB
        Steve Loughran
      5. HADOOP-9528-4.patch
        12 kB
        Steve Loughran
      6. HADOOP-9528-5.patch
        13 kB
        Steve Loughran
      7. HADOOP-9528-6.patch
        18 kB
        Steve Loughran
      8. HADOOP-9528-7.patch
        27 kB
        Steve Loughran

        Issue Links

          Activity

          Steve Loughran created issue -
          Steve Loughran made changes -
          Field Original Value New Value
          Link This issue contains HADOOP-9227 [ HADOOP-9227 ]
          Steve Loughran made changes -
          Link This issue contains HADOOP-9228 [ HADOOP-9228 ]
          Steve Loughran made changes -
          Link This issue contains HADOOP-9259 [ HADOOP-9259 ]
          Hide
          Steve Loughran added a comment -

          rolls up

          1. HADOOP-9227 FileSystemContractBaseTest doesn't test filesystem's mkdir/isDirectory() logic rigorously enough
          2. HADOOP-9228 FileSystemContractTestBase never verifies that files are files
          3. HADOOP-9259 FileSystemContractBaseTest should be less brittle in teardown

          Adds

          1. you can't rename a directory to be a child of itself or a child of a child directory.
          2. you can't rename the root directory.
          Show
          Steve Loughran added a comment - rolls up HADOOP-9227 FileSystemContractBaseTest doesn't test filesystem's mkdir/isDirectory() logic rigorously enough HADOOP-9228 FileSystemContractTestBase never verifies that files are files HADOOP-9259 FileSystemContractBaseTest should be less brittle in teardown Adds you can't rename a directory to be a child of itself or a child of a child directory. you can't rename the root directory.
          Hide
          Steve Loughran added a comment -

          Single integrated patch for stricter checks on filesystems.

          Show
          Steve Loughran added a comment - Single integrated patch for stricter checks on filesystems.
          Steve Loughran made changes -
          Attachment HADOOP-9528.patch [ 12566825 ]
          Steve Loughran made changes -
          Link This issue blocks HADOOP-8545 [ HADOOP-8545 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566825/HADOOP-9528.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract
          org.apache.hadoop.fs.TestTrash

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2108//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2108//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566825/HADOOP-9528.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract org.apache.hadoop.fs.TestTrash +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2108//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2108//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          TestTrash failing on mkdir

          2013-01-28 22:25:42,002 WARN  fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(138)) - Can't create trash directory: file:/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/trunk/hadoop-common-project/hadoop-common/target/test/data/testTrash/user/test/.Trash/Current/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/trunk/hadoop-common-project/hadoop-common/target/test/data/testTrash/test/mkdirs
          
          Show
          Steve Loughran added a comment - TestTrash failing on mkdir 2013-01-28 22:25:42,002 WARN fs.TrashPolicyDefault (TrashPolicyDefault.java:moveToTrash(138)) - Can't create trash directory: file:/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/trunk/hadoop-common-project/hadoop-common/target/test/data/testTrash/user/test/.Trash/Current/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/trunk/hadoop-common-project/hadoop-common/target/test/data/testTrash/test/mkdirs
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          Updated patch, adds another part of the contract, that FileSystem.getFileStatus("/") must always return a valid entry. That is: there is always a root directory.

          The S3 Native tests that are failing (both in memory and tested against s3) break this requirement.

          Similarly, both S3 filesystems let you rename a directory into a child directory.

          Show
          Steve Loughran added a comment - Updated patch, adds another part of the contract, that FileSystem.getFileStatus("/") must always return a valid entry. That is: there is always a root directory. The S3 Native tests that are failing (both in memory and tested against s3) break this requirement. Similarly, both S3 filesystems let you rename a directory into a child directory.
          Steve Loughran made changes -
          Attachment HADOOP-9528-2.patch [ 12566884 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566884/HADOOP-9528-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2109//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2109//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566884/HADOOP-9528-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2109//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2109//console This message is automatically generated.
          Hide
          Tom White added a comment -

          Steve, good to see more tests. The S3 failures sound like bugs to me, so it should be possible to fix them without breaking applications. Do you agree?

          Are you planning on including the local FS too?

          Show
          Tom White added a comment - Steve, good to see more tests. The S3 failures sound like bugs to me, so it should be possible to fix them without breaking applications. Do you agree? Are you planning on including the local FS too?
          Hide
          Steve Loughran added a comment -

          Tom the S3 failures are bugs there, the rename one being more serious, as it will lose data (after the rename, the parent dir is deleted).

          I don't know enough about the code there to fix fast -can you do it?. FWIW, the swift:// test for child-ness was trivial:

            public static boolean isChildOf(SwiftObjectPath parent,
                                            SwiftObjectPath possibleChild) {
              return possibleChild.getObject().startsWith(parent.getObject() + "/");
            }
          

          I'm not going to do Local FS test with this once this is in I plan to write some Junit4 tests alongside this contract to be even more rigorous.

          Show
          Steve Loughran added a comment - Tom the S3 failures are bugs there, the rename one being more serious, as it will lose data (after the rename, the parent dir is deleted). I don't know enough about the code there to fix fast -can you do it?. FWIW, the swift:// test for child-ness was trivial: public static boolean isChildOf(SwiftObjectPath parent, SwiftObjectPath possibleChild) { return possibleChild.getObject().startsWith(parent.getObject() + "/" ); } I'm not going to do Local FS test with this once this is in I plan to write some Junit4 tests alongside this contract to be even more rigorous.
          Steve Loughran made changes -
          Link This issue is related to HADOOP-9261 [ HADOOP-9261 ]
          Steve Loughran made changes -
          Link This issue breaks HADOOP-9265 [ HADOOP-9265 ]
          Hide
          Steve Loughran added a comment -

          HADOOP-9265 is the issue covering the s3:// failure

          Show
          Steve Loughran added a comment - HADOOP-9265 is the issue covering the s3:// failure
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          Updated patch

          1. allows subclasses to turn of case sensitivity tests
          2. verifies that any subdir checking logic doesn't cause renames to a dest with the same initial pathname as the src to fail
          Show
          Steve Loughran added a comment - Updated patch allows subclasses to turn of case sensitivity tests verifies that any subdir checking logic doesn't cause renames to a dest with the same initial pathname as the src to fail
          Steve Loughran made changes -
          Attachment HADOOP-9528-3.patch [ 12567193 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12567193/HADOOP-9528-3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2119//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2119//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12567193/HADOOP-9528-3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2119//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2119//console This message is automatically generated.
          Hide
          Tom White added a comment -

          +1 once the S3 fixes have been committed.

          Show
          Tom White added a comment - +1 once the S3 fixes have been committed.
          Hide
          Steve Loughran added a comment -

          I've just been making some minor changes to make it easier for subclasses to skip the case tests, and some more tests for HDFS-4332 -though they don't replicate the problem.

          I'll push this patch out today for review.

          -steve

          Show
          Steve Loughran added a comment - I've just been making some minor changes to make it easier for subclasses to skip the case tests, and some more tests for HDFS-4332 -though they don't replicate the problem. I'll push this patch out today for review. -steve
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Link This issue is related to HADOOP-9261 [ HADOOP-9261 ]
          Steve Loughran made changes -
          Link This issue breaks HADOOP-9261 [ HADOOP-9261 ]
          Hide
          Steve Loughran added a comment -
          1. adds test for listing files in /
          2. adds tests for case sensitivity (which subclasses may disable).
          3. adds tests for renaming onto self for file & dir.
          4. adds tests for renaming into parent directory (which devolves for renaming onto self, and so becomes a success)

          Return codes expected from the renames are the same as HDFS (verified with the HDFS tests), which are those of Unix.

          Breaks the previous patch for HADOOP-9261 as that didn't handle the rename into parent dir.

          Show
          Steve Loughran added a comment - adds test for listing files in / adds tests for case sensitivity (which subclasses may disable). adds tests for renaming onto self for file & dir. adds tests for renaming into parent directory (which devolves for renaming onto self, and so becomes a success) Return codes expected from the renames are the same as HDFS (verified with the HDFS tests), which are those of Unix. Breaks the previous patch for HADOOP-9261 as that didn't handle the rename into parent dir.
          Steve Loughran made changes -
          Attachment HADOOP-9528-4.patch [ 12567685 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12567685/HADOOP-9528-4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2136//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2136//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12567685/HADOOP-9528-4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2136//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2136//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          cancelling mv(self, self) return codes may be not fully in sync w/ posix -need to verify more & check specs before defining tests.

          Show
          Steve Loughran added a comment - cancelling mv(self, self) return codes may be not fully in sync w/ posix -need to verify more & check specs before defining tests.
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          patch w/ more robust teardown and rename return codes for rename(self, self) matching HDFS & posix expectations.

          Show
          Steve Loughran added a comment - patch w/ more robust teardown and rename return codes for rename(self, self) matching HDFS & posix expectations.
          Steve Loughran made changes -
          Attachment HADOOP-9528-5.patch [ 12570828 ]
          Hide
          Steve Loughran added a comment -

          submit for a test run

          Show
          Steve Loughran added a comment - submit for a test run
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570828/HADOOP-9528-5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2230//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2230//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570828/HADOOP-9528-5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2230//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2230//console This message is automatically generated.
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          This patch incorporates the HADOOP-9261 s3n rename changes.

          Show
          Steve Loughran added a comment - This patch incorporates the HADOOP-9261 s3n rename changes.
          Steve Loughran made changes -
          Attachment HADOOP-9528-6.patch [ 12570974 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Link This issue contains HADOOP-9261 [ HADOOP-9261 ]
          Steve Loughran made changes -
          Link This issue breaks HADOOP-9261 [ HADOOP-9261 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570974/HADOOP-9528-6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2231//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2231//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570974/HADOOP-9528-6.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2231//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2231//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          The test still failing depends on HADOOP-9265

          Show
          Steve Loughran added a comment - The test still failing depends on HADOOP-9265
          Steve Loughran made changes -
          Parent HADOOP-9361 [ 12635325 ]
          Issue Type Improvement [ 4 ] Sub-task [ 7 ]
          Hide
          Matt Foley added a comment -

          Based on Tom White's +1, can commit after HADOOP-9265 – which seems ready to commit.

          Show
          Matt Foley added a comment - Based on Tom White's +1, can commit after HADOOP-9265 – which seems ready to commit.
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -
          1. this patch incorporates HADOOP-9265
          2. ignores the return value of rename(self,self) when self.isDirectory(), as HDFS returns success where posix returns false
          Show
          Steve Loughran added a comment - this patch incorporates HADOOP-9265 ignores the return value of rename(self,self) when self.isDirectory() , as HDFS returns success where posix returns false
          Steve Loughran made changes -
          Attachment HADOOP-9528-7.patch [ 12573102 ]
          Hide
          Steve Loughran added a comment -

          No @Test timeout attributes as this is a JUnit3 test case.

          Show
          Steve Loughran added a comment - No @Test timeout attributes as this is a JUnit3 test case.
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Link This issue contains HADOOP-9265 [ HADOOP-9265 ]
          Steve Loughran made changes -
          Link This issue breaks HADOOP-9265 [ HADOOP-9265 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12573102/HADOOP-9528-7.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12573102/HADOOP-9528-7.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          Note that the real S3 tests need to be verified by explicitly running them

          Show
          Steve Loughran added a comment - Note that the real S3 tests need to be verified by explicitly running them
          Hide
          Tom White added a comment -

          I tried running Jets3tS3FileSystemContractTest by setting the following properties in src/test/resources/core-site.xml:

          <property>
            <name>test.fs.s3.name</name>
            <value>s3://mytestbucket</value>
            <description>The name of the s3 file system for testing.</description>
          </property>
          
          <property>
            <name>fs.s3.awsAccessKeyId</name>
            <value>xxx</value>
          </property>
          
          <property>
            <name>fs.s3.awsSecretAccessKey</name>
            <value>xxx</value>
          </property>
          

          However, testLSRootDir is consistently hanging with the following stacktrace.

          Steve, did you manage to run against S3 yet?

          "main" prio=5 tid=105000000 nid=0x100601000 runnable [1005fd000]
             java.lang.Thread.State: RUNNABLE
          	at java.net.SocketInputStream.socketRead0(Native Method)
          	at java.net.SocketInputStream.read(SocketInputStream.java:129)
          	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
          	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
          	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
          	- locked <7bb659288> (a java.lang.Object)
          	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820)
          	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
          	- locked <7bb659338> (a com.sun.net.ssl.internal.ssl.AppInputStream)
          	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
          	at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
          	- locked <7bb66ad28> (a java.io.BufferedInputStream)
          	at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
          	at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
          	at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
          	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
          	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
          	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
          	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
          	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
          	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
          	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
          	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
          	at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:357)
          	at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:686)
          	at org.jets3t.service.impl.rest.httpclient.RestS3Service.listObjectsInternal(RestS3Service.java:1083)
          	at org.jets3t.service.impl.rest.httpclient.RestS3Service.listObjectsImpl(RestS3Service.java:1046)
          	at org.jets3t.service.S3Service.listObjects(S3Service.java:1299)
          	at org.jets3t.service.S3Service.listObjects(S3Service.java:1271)
          	at org.jets3t.service.S3Service.listObjects(S3Service.java:1137)
          	at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.listSubPaths(Jets3tFileSystemStore.java:279)
          	at org.apache.hadoop.fs.s3.S3FileSystem.listStatus(S3FileSystem.java:202)
          	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1430)
          	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1470)
          	at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:1745)
          	at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1744)
          	at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1727)
          	at org.apache.hadoop.fs.FileSystem$5.handleFileStat(FileSystem.java:1820)
          	at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1797)
          	at org.apache.hadoop.fs.FileSystemContractBaseTest.assertListFilesFinds(FileSystemContractBaseTest.java:719)
          	at org.apache.hadoop.fs.FileSystemContractBaseTest.testLSRootDir(FileSystemContractBaseTest.java:704)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at junit.framework.TestCase.runTest(TestCase.java:168)
          	at junit.framework.TestCase.runBare(TestCase.java:134)
          	at junit.framework.TestResult$1.protect(TestResult.java:110)
          	at junit.framework.TestResult.runProtected(TestResult.java:128)
          	at junit.framework.TestResult.run(TestResult.java:113)
          	at junit.framework.TestCase.run(TestCase.java:124)
          	at junit.framework.TestSuite.runTest(TestSuite.java:243)
          	at junit.framework.TestSuite.run(TestSuite.java:238)
          	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
          	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
          	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
          	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
          	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
          
          Show
          Tom White added a comment - I tried running Jets3tS3FileSystemContractTest by setting the following properties in src/test/resources/core-site.xml: <property> <name>test.fs.s3.name</name> <value>s3://mytestbucket</value> <description>The name of the s3 file system for testing.</description> </property> <property> <name>fs.s3.awsAccessKeyId</name> <value>xxx</value> </property> <property> <name>fs.s3.awsSecretAccessKey</name> <value>xxx</value> </property> However, testLSRootDir is consistently hanging with the following stacktrace. Steve, did you manage to run against S3 yet? "main" prio=5 tid=105000000 nid=0x100601000 runnable [1005fd000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293) at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863) - locked <7bb659288> (a java.lang.Object) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820) at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75) - locked <7bb659338> (a com.sun.net.ssl.internal.ssl.AppInputStream) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) - locked <7bb66ad28> (a java.io.BufferedInputStream) at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116) at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413) at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973) at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:357) at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:686) at org.jets3t.service.impl.rest.httpclient.RestS3Service.listObjectsInternal(RestS3Service.java:1083) at org.jets3t.service.impl.rest.httpclient.RestS3Service.listObjectsImpl(RestS3Service.java:1046) at org.jets3t.service.S3Service.listObjects(S3Service.java:1299) at org.jets3t.service.S3Service.listObjects(S3Service.java:1271) at org.jets3t.service.S3Service.listObjects(S3Service.java:1137) at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.listSubPaths(Jets3tFileSystemStore.java:279) at org.apache.hadoop.fs.s3.S3FileSystem.listStatus(S3FileSystem.java:202) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1430) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1470) at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:1745) at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1744) at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1727) at org.apache.hadoop.fs.FileSystem$5.handleFileStat(FileSystem.java:1820) at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1797) at org.apache.hadoop.fs.FileSystemContractBaseTest.assertListFilesFinds(FileSystemContractBaseTest.java:719) at org.apache.hadoop.fs.FileSystemContractBaseTest.testLSRootDir(FileSystemContractBaseTest.java:704) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
          Hide
          Steve Loughran added a comment -

          I thought I'd run against S3 & S3n all the tests, but trying today on S3 I see the same stack trace. cancelling the patch until I know more.

          Show
          Steve Loughran added a comment - I thought I'd run against S3 & S3n all the tests, but trying today on S3 I see the same stack trace. cancelling the patch until I know more.
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          resubmitting -I think it is S3-from-the-UK that isn't working today, as other tools of mine are failing, and this test did work yesterday

          Show
          Steve Loughran added a comment - resubmitting -I think it is S3-from-the-UK that isn't working today, as other tools of mine are failing, and this test did work yesterday
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Steve Loughran added a comment -

          removing one test that plays up on S3

          Show
          Steve Loughran added a comment - removing one test that plays up on S3
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Link This issue is related to HADOOP-9410 [ HADOOP-9410 ]
          Hide
          Steve Loughran added a comment -

          8th update, this time with the correct JIRA number in its name

          Show
          Steve Loughran added a comment - 8th update, this time with the correct JIRA number in its name
          Steve Loughran made changes -
          Attachment HADOOP-9258-8.patch [ 12574074 ]
          Hide
          Steve Loughran added a comment -

          The latest patch strips out the test failing against S3, which is now in HADOOP-9410. That JIRA shows the problem exists even when the changes to S3 rename are not applied, showing that it is independent of the patch here.

          As that test is only applicable to Hadoop 2.x, this patch addresses all the issues found for branch 1.x, to which it can also be applied

          Show
          Steve Loughran added a comment - The latest patch strips out the test failing against S3, which is now in HADOOP-9410 . That JIRA shows the problem exists even when the changes to S3 rename are not applied, showing that it is independent of the patch here. As that test is only applicable to Hadoop 2.x, this patch addresses all the issues found for branch 1.x, to which it can also be applied
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12574074/HADOOP-9258-8.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 tests included appear to have a timeout.

          -1 javac. The applied patch generated 1364 javac compiler warnings (more than the trunk's current 1363 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574074/HADOOP-9258-8.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 tests included appear to have a timeout. -1 javac . The applied patch generated 1364 javac compiler warnings (more than the trunk's current 1363 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2335//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          javac warning is about use of isDir(). isDir() may be deprecated in trunk, bit isDirectory() isn't in branch-1, and these tests (and S3) patches are designed to be backported

          Show
          Steve Loughran added a comment - javac warning is about use of isDir() . isDir() may be deprecated in trunk, bit isDirectory() isn't in branch-1, and these tests (and S3) patches are designed to be backported
          Hide
          Tom White added a comment -

          The live tests now pass for me. +1

          Show
          Tom White added a comment - The live tests now pass for me. +1
          Hide
          Steve Loughran added a comment -

          committed in to trunk. Tom -thanks your input, reviews and your involvement in the s3 & s3n patches!

          Show
          Steve Loughran added a comment - committed in to trunk. Tom -thanks your input, reviews and your involvement in the s3 & s3n patches!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3520 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3520/)
          HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228, HADOOP-9227 & HADOOP-9259) (Revision 1460646)

          Result = SUCCESS
          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3520 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3520/ ) HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228 , HADOOP-9227 & HADOOP-9259 ) (Revision 1460646) Result = SUCCESS stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/167/)
          HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228, HADOOP-9227 & HADOOP-9259) (Revision 1460646)

          Result = SUCCESS
          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/167/ ) HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228 , HADOOP-9227 & HADOOP-9259 ) (Revision 1460646) Result = SUCCESS stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1356 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1356/)
          HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228, HADOOP-9227 & HADOOP-9259) (Revision 1460646)

          Result = FAILURE
          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1356 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1356/ ) HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228 , HADOOP-9227 & HADOOP-9259 ) (Revision 1460646) Result = FAILURE stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1384/)
          HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228, HADOOP-9227 & HADOOP-9259) (Revision 1460646)

          Result = FAILURE
          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1384/ ) HADOOP-9258 Add stricter tests to FileSystemContractTestBase (includes fixes for production code HADOOP-9261 & HADOOP-9265 and test enhancements HADOOP-9228 , HADOOP-9227 & HADOOP-9259 ) (Revision 1460646) Result = FAILURE stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460646 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/S3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3/InMemoryFileSystemStore.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Gavin made changes -
          Link This issue blocks HADOOP-8545 [ HADOOP-8545 ]
          Gavin made changes -
          Link This issue is depended upon by HADOOP-8545 [ HADOOP-8545 ]
          Hide
          Suresh Srinivas added a comment -

          Steve Loughran Steve, I suggest merging this to branch-2. I think this should also go into branch-2.1.0-beta.

          Show
          Suresh Srinivas added a comment - Steve Loughran Steve, I suggest merging this to branch-2. I think this should also go into branch-2.1.0-beta.
          Hide
          Steve Loughran added a comment -

          go for it!

          Show
          Steve Loughran added a comment - go for it!

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development