Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None
    • Release Note:
      Adds support for Parascale storage.

      Description

      Parascale has developed an org.apache.hadoop.fs implementation that allows users to use Hadoop on Parascale storage clusters. We'd like to contribute this work to the community. Should this be placed under contrib, or integrated into the org.apache.hadoop.fs space?

      1. HADOOP-6704_0_20_2-3.patch
        163 kB
        Neil Bliss
      2. HADOOP-6704-10.patch
        129 kB
        Neil Bliss
      3. HADOOP-6704-8.patch
        129 kB
        Neil Bliss
      4. HADOOP-6704_0_20_2-2.patch
        160 kB
        Neil Bliss
      5. HADOOP-6704-7.patch
        126 kB
        Neil Bliss
      6. HADOOP-6704-6.patch
        126 kB
        Neil Bliss
      7. HADOOP-6704-5.patch
        126 kB
        Neil Bliss
      8. HADOOP-6704-4.patch
        126 kB
        Neil Bliss
      9. HADOOP-6704-3.patch
        126 kB
        Neil Bliss
      10. HADOOP-6704-2.patch
        126 kB
        Neil Bliss
      11. HADOOP-6704.patch
        126 kB
        Neil Bliss
      12. HADOOP-6704.0.20.2.patch
        158 kB
        Neil Bliss

        Activity

        Hide
        Eli Collins added a comment -

        Seems like it should go in o.a.h.fs with the other non-hdfs file systems if it's going to be actively maintained. Will need some unit tests to help prevent people from breaking it since hudson doesn't have a parascale backend.

        Show
        Eli Collins added a comment - Seems like it should go in o.a.h.fs with the other non-hdfs file systems if it's going to be actively maintained. Will need some unit tests to help prevent people from breaking it since hudson doesn't have a parascale backend.
        Hide
        Neil Bliss added a comment -

        ok, patch forthcoming shortly.

        Show
        Neil Bliss added a comment - ok, patch forthcoming shortly.
        Hide
        Steve Loughran added a comment -
        1. This is a new feature, so the patch should be against SVN_HEAD, not an older version of Hadoop.
        2. The newer version of Hadoop is moving to some new FS APIs; you might want to consider working with them.
        3. HDFS-708 has discussed the issue of stress testing filesystems; this new filesystem back end could be a use case, if you are willing to participate.
        4. One issue with all third party filesystems is regression testing: they don't get enough of it. If there is any way to make this easier -and that could include you running a local version of Hudson to grab SVN_HEAD of Hadoop and testing MR jobs over your filestore, then end users will be grateful.
        5. Involvement in testing forthcoming releases is equally important, as is ongoing maintenance. It is really hard for an OSS project to test/maintain code that works with other peoples infrastructure, and motivation can be trouble too, so you have to be willing to stay involved -otherwise the code just gradually stops working.
        Show
        Steve Loughran added a comment - This is a new feature, so the patch should be against SVN_HEAD, not an older version of Hadoop. The newer version of Hadoop is moving to some new FS APIs; you might want to consider working with them. HDFS-708 has discussed the issue of stress testing filesystems; this new filesystem back end could be a use case, if you are willing to participate. One issue with all third party filesystems is regression testing: they don't get enough of it. If there is any way to make this easier -and that could include you running a local version of Hudson to grab SVN_HEAD of Hadoop and testing MR jobs over your filestore, then end users will be grateful. Involvement in testing forthcoming releases is equally important, as is ongoing maintenance. It is really hard for an OSS project to test/maintain code that works with other peoples infrastructure, and motivation can be trouble too, so you have to be willing to stay involved -otherwise the code just gradually stops working.
        Hide
        Neil Bliss added a comment -

        Here's a patch against the 0.20.2 tag. A patch against trunk will be forthcoming shortly.

        I'll take a look at the stress testing Jira and see if it's something we can help with.

        Show
        Neil Bliss added a comment - Here's a patch against the 0.20.2 tag. A patch against trunk will be forthcoming shortly. I'll take a look at the stress testing Jira and see if it's something we can help with.
        Hide
        Neil Bliss added a comment -

        that patch still contains unit test failures. I will fix it and respin the patch.

        Show
        Neil Bliss added a comment - that patch still contains unit test failures. I will fix it and respin the patch.
        Hide
        Neil Bliss added a comment -

        Patches to -0.20.2 and -trunk adding support for Parascale storage as the DFS.

        Show
        Neil Bliss added a comment - Patches to -0.20.2 and -trunk adding support for Parascale storage as the DFS.
        Hide
        Neil Bliss added a comment -

        Adds support for Parascale storage as the backing store for Hadoop instances.

        Show
        Neil Bliss added a comment - Adds support for Parascale storage as the backing store for Hadoop instances.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443204/HADOOP-6704.patch
        against trunk revision 939510.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 61 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1025 javac compiler warnings (more than the trunk's current 1017 warnings).

        -1 findbugs. The patch appears to introduce 2 new Findbugs warnings.

        -1 release audit. The applied patch generated 5 release audit warnings (more than the trunk's current 1 warnings).

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443204/HADOOP-6704.patch against trunk revision 939510. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 61 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1025 javac compiler warnings (more than the trunk's current 1017 warnings). -1 findbugs. The patch appears to introduce 2 new Findbugs warnings. -1 release audit. The applied patch generated 5 release audit warnings (more than the trunk's current 1 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/490/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        Second attempt at patch submission.

        Show
        Neil Bliss added a comment - Second attempt at patch submission.
        Hide
        Neil Bliss added a comment -

        Second attempt for patch submission.

        Show
        Neil Bliss added a comment - Second attempt for patch submission.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443345/HADOOP-6704-2.patch
        against trunk revision 939864.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 61 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings).

        -1 findbugs. The patch appears to introduce 3 new Findbugs warnings.

        -1 release audit. The applied patch generated 5 release audit warnings (more than the trunk's current 1 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443345/HADOOP-6704-2.patch against trunk revision 939864. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 61 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings). -1 findbugs. The patch appears to introduce 3 new Findbugs warnings. -1 release audit. The applied patch generated 5 release audit warnings (more than the trunk's current 1 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/64/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        The -3 version of this patch addresses findbugs, releaseaudit, and checkstyle warnings that were present in the -2 version.

        Show
        Neil Bliss added a comment - The -3 version of this patch addresses findbugs, releaseaudit, and checkstyle warnings that were present in the -2 version.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443588/HADOOP-6704-3.patch
        against trunk revision 940527.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings).

        -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443588/HADOOP-6704-3.patch against trunk revision 940527. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings). -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/498/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        I now see the problem resulting in the findbugs warning. Fixing it now.

        Show
        Neil Bliss added a comment - I now see the problem resulting in the findbugs warning. Fixing it now.
        Hide
        Neil Bliss added a comment -

        -3 patch still contains a findbugs warning. This will be addressed in the -4 version of the patch.

        Show
        Neil Bliss added a comment - -3 patch still contains a findbugs warning. This will be addressed in the -4 version of the patch.
        Hide
        Neil Bliss added a comment -

        The -4 version of this patch addresses the findbugs warning from the -3 version, specifically it adds a ChunkInfo.equals() implementation.

        Show
        Neil Bliss added a comment - The -4 version of this patch addresses the findbugs warning from the -3 version, specifically it adds a ChunkInfo.equals() implementation.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443598/HADOOP-6704-4.patch
        against trunk revision 940527.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings).

        -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443598/HADOOP-6704-4.patch against trunk revision 940527. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings). -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/499/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        I ran findbugs locally before submitting the -4 version of the patch, and it came back clean. I can't help but wonder why I'm seeing different results from Hudson. The findbugs complaint is now that I don't have ChunkInfo.hashCode() implemented. I'll implement that, and resubmit to see what Hudson has to say then.

        Show
        Neil Bliss added a comment - I ran findbugs locally before submitting the -4 version of the patch, and it came back clean. I can't help but wonder why I'm seeing different results from Hudson. The findbugs complaint is now that I don't have ChunkInfo.hashCode() implemented. I'll implement that, and resubmit to see what Hudson has to say then.
        Hide
        Neil Bliss added a comment -

        ahhh, findbugs is an external dependency that's not installed on my system. Running "ant findbugs" without having it installed doesn't complain that it's not there, so I didn't know I was missing the tool.

        Show
        Neil Bliss added a comment - ahhh, findbugs is an external dependency that's not installed on my system. Running "ant findbugs" without having it installed doesn't complain that it's not there, so I didn't know I was missing the tool.
        Hide
        Neil Bliss added a comment -

        Version -5 of this patch. Addresses all found findbugs and checkstyle warnings. Adds ChunkInfo.hashCode() implementation.

        Show
        Neil Bliss added a comment - Version -5 of this patch. Addresses all found findbugs and checkstyle warnings. Adds ChunkInfo.hashCode() implementation.
        Hide
        Neil Bliss added a comment -

        Trying again.

        Show
        Neil Bliss added a comment - Trying again.
        Hide
        Neil Bliss added a comment -

        I think I goofed up the order of operations in submitting the -5 version of the patch. I'm going to resubmit it.

        Show
        Neil Bliss added a comment - I think I goofed up the order of operations in submitting the -5 version of the patch. I'm going to resubmit it.
        Hide
        Neil Bliss added a comment -

        Resubmit of the -5 version of this patch. Adds ChunkInfo.hashCode().

        Show
        Neil Bliss added a comment - Resubmit of the -5 version of this patch. Adds ChunkInfo.hashCode().
        Hide
        Neil Bliss added a comment -

        Second try at submitting the -5 version of this patch.

        Show
        Neil Bliss added a comment - Second try at submitting the -5 version of this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443654/HADOOP-6704-5.patch
        against trunk revision 940989.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings).

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443654/HADOOP-6704-5.patch against trunk revision 940989. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1022 javac compiler warnings (more than the trunk's current 1017 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/501/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        This latest test failure comes from 5 javac warnings about the use of a deprecated interface to org.apache.hadoop.fs.Path.makeQualified(). The deprecation message in the source indicates to use makeQualified(URI, Path) instead of makeQualified(FileSystem). In looking through the rest of the codebase, however, every reference to makeQualified is using the deprecated interface. How robust is the new interface, and is the old interface really deprecated?

        Show
        Neil Bliss added a comment - This latest test failure comes from 5 javac warnings about the use of a deprecated interface to org.apache.hadoop.fs.Path.makeQualified(). The deprecation message in the source indicates to use makeQualified(URI, Path) instead of makeQualified(FileSystem). In looking through the rest of the codebase, however, every reference to makeQualified is using the deprecated interface. How robust is the new interface, and is the old interface really deprecated?
        Hide
        Neil Bliss added a comment -

        Version -6 of this patch, revises org.apache.hadoop.fs.Path.makeQualified() references to use non-deprecated interfaces.

        Show
        Neil Bliss added a comment - Version -6 of this patch, revises org.apache.hadoop.fs.Path.makeQualified() references to use non-deprecated interfaces.
        Hide
        Neil Bliss added a comment -

        If at first you don't succeed, try, try again.

        Show
        Neil Bliss added a comment - If at first you don't succeed, try, try again.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443664/HADOOP-6704-6.patch
        against trunk revision 940989.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 1018 javac compiler warnings (more than the trunk's current 1017 warnings).

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443664/HADOOP-6704-6.patch against trunk revision 940989. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1018 javac compiler warnings (more than the trunk's current 1017 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/502/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        Overlooked one more javac warning. Patch -7 forthcoming.

        Show
        Neil Bliss added a comment - Overlooked one more javac warning. Patch -7 forthcoming.
        Hide
        Neil Bliss added a comment -

        Version -7 of this patch. Remove the deprecated getName() interface from RawParascaleFileSystem.java, and remove the corresponding test case.

        Show
        Neil Bliss added a comment - Version -7 of this patch. Remove the deprecated getName() interface from RawParascaleFileSystem.java, and remove the corresponding test case.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443679/HADOOP-6704-7.patch
        against trunk revision 940989.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443679/HADOOP-6704-7.patch against trunk revision 940989. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/503/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704_0_20_2-2.patch applies the funtionality of HADOOP-6704-7.patch to the 0.20.2 branch.

        Show
        Neil Bliss added a comment - HADOOP-6704 _0_20_2-2.patch applies the funtionality of HADOOP-6704 -7.patch to the 0.20.2 branch.
        Hide
        Neil Bliss added a comment -

        We have a few more in-house changes to merge into this patch. Patch version -8 forthcoming shortly.

        Show
        Neil Bliss added a comment - We have a few more in-house changes to merge into this patch. Patch version -8 forthcoming shortly.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704-8.patch cleans up the testing infrastructure slightly and more completely cleans up after test runs.

        Show
        Neil Bliss added a comment - HADOOP-6704 -8.patch cleans up the testing infrastructure slightly and more completely cleans up after test runs.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704-8.patch incorporates improvements in the parascale unit tests.

        Show
        Neil Bliss added a comment - HADOOP-6704 -8.patch incorporates improvements in the parascale unit tests.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443914/HADOOP-6704-8.patch
        against trunk revision 941662.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 55 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443914/HADOOP-6704-8.patch against trunk revision 941662. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 55 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/509/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        Is there a next step for this?

        Show
        Neil Bliss added a comment - Is there a next step for this?
        Hide
        Eli Collins added a comment -

        Someone needs to review the patch, apologies for the slow response time, I think people are pretty swamped with internal stuff and blockers for the 21 release.

        Show
        Eli Collins added a comment - Someone needs to review the patch, apologies for the slow response time, I think people are pretty swamped with internal stuff and blockers for the 21 release.
        Hide
        Neil Bliss added a comment -

        I understand completely! I was just checking to make sure there wasn't something I was supposed to be doing at this point. Carry on!

        Show
        Neil Bliss added a comment - I understand completely! I was just checking to make sure there wasn't something I was supposed to be doing at this point. Carry on!
        Hide
        Neil Bliss added a comment -

        There have been a few bugfixes made on the internal development tree, so I'll withdraw the -8 patch and submit a newer one with the current code.

        Show
        Neil Bliss added a comment - There have been a few bugfixes made on the internal development tree, so I'll withdraw the -8 patch and submit a newer one with the current code.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704-9.patch enables usage of Parascale storage. The -9 version of this patch contains improvements to the cleanup after unit tests have been run, and an important bugfix in the HTTPChunkLocator code affecting operation in multithreaded client code.

        Show
        Neil Bliss added a comment - HADOOP-6704 -9.patch enables usage of Parascale storage. The -9 version of this patch contains improvements to the cleanup after unit tests have been run, and an important bugfix in the HTTPChunkLocator code affecting operation in multithreaded client code.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704-10.patch contains the same fixes described in the (since removed) -9 patch.

        Show
        Neil Bliss added a comment - HADOOP-6704 -10.patch contains the same fixes described in the (since removed) -9 patch.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12445159/HADOOP-6704-10.patch
        against trunk revision 946976.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 53 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445159/HADOOP-6704-10.patch against trunk revision 946976. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 53 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/537/console This message is automatically generated.
        Hide
        Neil Bliss added a comment -

        HADOOP-6704_0_20_2-3.patch provides important bugfixes over the -2 patch for the 0.20.2 branch.

        Show
        Neil Bliss added a comment - HADOOP-6704 _0_20_2-3.patch provides important bugfixes over the -2 patch for the 0.20.2 branch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12445180/HADOOP-6704_0_20_2-3.patch
        against trunk revision 1031422.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 95 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/40//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445180/HADOOP-6704_0_20_2-3.patch against trunk revision 1031422. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 95 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/40//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12445180/HADOOP-6704_0_20_2-3.patch
        against trunk revision 1071364.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 95 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/276//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445180/HADOOP-6704_0_20_2-3.patch against trunk revision 1071364. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 95 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/276//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        Am not sure if we still accept alternate FSes into the upstream because of the support it'd require even as a contrib module.

        Perhaps you'd be willing to instead continue carrying this on a github or apache-extras repository, outside of Apache Hadoop?

        Show
        Harsh J added a comment - Am not sure if we still accept alternate FSes into the upstream because of the support it'd require even as a contrib module. Perhaps you'd be willing to instead continue carrying this on a github or apache-extras repository, outside of Apache Hadoop?
        Hide
        Eli Collins added a comment -

        Closing out as stale, this issue hasn't been updated in over a year.

        Show
        Eli Collins added a comment - Closing out as stale, this issue hasn't been updated in over a year.

          People

          • Assignee:
            Unassigned
            Reporter:
            Neil Bliss
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 168h
              168h
              Remaining:
              Remaining Estimate - 168h
              168h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development