Hadoop Common
  1. Hadoop Common
  2. HADOOP-4234

KFS: Allow KFS layer to interface with multiple KFS namenodes

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.20.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Changed KFS glue layer to allow applications to interface with multiple KFS metaservers.

      Description

      The KFS "glue" layer code in Hadoop, currently, only allows an application to interface with a single KFS namenode. The KFS client side library has been modified to allow applications to interface with multiple KFS namenodes. This jira issue is for incorporating the change into the KFS code.

      1. patch.4234
        1 kB
        Sriram Rao

        Activity

        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #615 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/615/ )
        Hide
        Lohit Vijayarenu added a comment -

        The change comment for this should probably be in new feature or improvement rather than bug.

        Thanks Owen. Changed it with a comment.

        Show
        Lohit Vijayarenu added a comment - The change comment for this should probably be in new feature or improvement rather than bug. Thanks Owen. Changed it with a comment.
        Hide
        Owen O'Malley added a comment -

        The change comment for this should probably be in new feature or improvement rather than bug.

        Show
        Owen O'Malley added a comment - The change comment for this should probably be in new feature or improvement rather than bug.
        Hide
        Lohit Vijayarenu added a comment -

        contrib test failure is tracked in HADOOP-4257. And test cases are not included as this was tested on KFS deployment.

        Show
        Lohit Vijayarenu added a comment - contrib test failure is tracked in HADOOP-4257 . And test cases are not included as this was tested on KFS deployment.
        Hide
        Lohit Vijayarenu added a comment -

        I just committed this. Thank you Sriram.

        Show
        Lohit Vijayarenu added a comment - I just committed this. Thank you Sriram.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12390805/patch.4234
        against trunk revision 698385.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390805/patch.4234 against trunk revision 698385. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3359/console This message is automatically generated.
        Hide
        Sriram Rao added a comment -

        Updated patch that passes the tests.

        Show
        Sriram Rao added a comment - Updated patch that passes the tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12390719/patch.4234
        against trunk revision 698187.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390719/patch.4234 against trunk revision 698187. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3355/console This message is automatically generated.
        Hide
        Sriram Rao added a comment -

        Providing a new patch since the previous one didn't work.

        Re: no new test, this issue was found in a KFS deployment. To test this code path, a KFS+Hadoop deployment is required; that was done elsewhere.

        Show
        Sriram Rao added a comment - Providing a new patch since the previous one didn't work. Re: no new test, this issue was found in a KFS deployment. To test this code path, a KFS+Hadoop deployment is required; that was done elsewhere.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12390672/patch.4234
        against trunk revision 697306.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3347/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390672/patch.4234 against trunk revision 697306. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3347/console This message is automatically generated.
        Hide
        Sriram Rao added a comment -

        @Tsz: In the patch that is submitted, in the Init() method, we don't need to check whether kfsImpl == null. The KFS client library code keeps track of all the servers to which the client is connected to; it build a new "handle" for each new server.

        Show
        Sriram Rao added a comment - @Tsz: In the patch that is submitted, in the Init() method, we don't need to check whether kfsImpl == null. The KFS client library code keeps track of all the servers to which the client is connected to; it build a new "handle" for each new server.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Should we keep checking whether kfsImpl == null?

        Show
        Tsz Wo Nicholas Sze added a comment - Should we keep checking whether kfsImpl == null?
        Hide
        Lohit Vijayarenu added a comment -

        +1 Looks good.

        Show
        Lohit Vijayarenu added a comment - +1 Looks good.
        Hide
        Sriram Rao added a comment -

        A patch file that fixes the issue is attached. This issue was found in a KFS deployment and was tested there. Testing this fix requires a KFS deployment.

        Show
        Sriram Rao added a comment - A patch file that fixes the issue is attached. This issue was found in a KFS deployment and was tested there. Testing this fix requires a KFS deployment.

          People

          • Assignee:
            Sriram Rao
            Reporter:
            Sriram Rao
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development