Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-930

o.a.h.hdfs.server.datanode.DataXceiver - run() - Version mismatch exception - more context to help debugging

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      add some context information in the IOException during a version mismatch to help debugging.

      (Applicable on the 0.20.x branch )

      1. HADOOP-6519.patch
        0.8 kB
        Karthik K
      2. HDFS-930.patch
        0.8 kB
        Karthik K
      3. HDFS-930.patch
        0.7 kB
        Karthik K

        Issue Links

          Activity

          Karthik K created issue -
          Karthik K made changes -
          Field Original Value New Value
          Affects Version/s 0.20.1 [ 12313866 ]
          Description add some context information in the IOException during a version mismatch to help debugging. add some context information in the IOException during a version mismatch to help debugging.

          (Applicable on the 0.20.x branch )
          Karthik K made changes -
          Attachment HADOOP-6519.patch [ 12431736 ]
          Karthik K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Todd Lipcon added a comment -

          Patch looks good to me. Can you do this on trunk as well? As this is an improvement, it should go in trunk and maybe some people will elect to backport to 0.20

          Show
          Todd Lipcon added a comment - Patch looks good to me. Can you do this on trunk as well? As this is an improvement, it should go in trunk and maybe some people will elect to backport to 0.20
          Hide
          Karthik K added a comment -

          Sure - so i create a separate jira for trunk as well ? Is this part supposed to have moved to HDFS tree ?

          Show
          Karthik K added a comment - Sure - so i create a separate jira for trunk as well ? Is this part supposed to have moved to HDFS tree ?
          Hide
          Todd Lipcon added a comment -

          You can use the samis JIRA for trunk - just submit a patch here for trunk. The protocol is to fix bugs in trunk, then later discuss whether to backport.

          Since this is an HDFS improvement, I'll move the jira over to the HDFS project.

          Show
          Todd Lipcon added a comment - You can use the samis JIRA for trunk - just submit a patch here for trunk. The protocol is to fix bugs in trunk, then later discuss whether to backport. Since this is an HDFS improvement, I'll move the jira over to the HDFS project.
          Todd Lipcon made changes -
          Project Hadoop Common [ 12310240 ] Hadoop HDFS [ 12310942 ]
          Key HADOOP-6519 HDFS-930
          Affects Version/s 0.22.0 [ 12314241 ]
          Affects Version/s 0.20.1 [ 12313866 ]
          Hide
          Karthik K added a comment -

          @Todd thanks for the clarification. Will submit a separate patch for trunk. Current patch in its current form does not apply to HDFS tree.

          Show
          Karthik K added a comment - @Todd thanks for the clarification. Will submit a separate patch for trunk. Current patch in its current form does not apply to HDFS tree.
          Karthik K made changes -
          Attachment HDFS-930.patch [ 12431887 ]
          Hide
          Karthik K added a comment -

          revised patch should apply properly in HDFS , HDFS-930 against trunk

          Show
          Karthik K added a comment - revised patch should apply properly in HDFS , HDFS-930 against trunk
          Hide
          Karthik K added a comment -

          Rationale: When playing around with different patched versions - version mismatch with the numbers help in making sure the installations are consistent with each other.

          Show
          Karthik K added a comment - Rationale: When playing around with different patched versions - version mismatch with the numbers help in making sure the installations are consistent with each other.
          Karthik K made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Karthik K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12431887/HDFS-930.patch
          against trunk revision 907964.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12431887/HDFS-930.patch against trunk revision 907964. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/113/console This message is automatically generated.
          Tsz Wo Nicholas Sze made changes -
          Link This issue duplicates HDFS-341 [ HDFS-341 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Patch look good.

          Could you reformat the new codes so that each line only takes <= 80 characters? It is the Hadoop code style.

          Show
          Tsz Wo Nicholas Sze added a comment - Patch look good. Could you reformat the new codes so that each line only takes <= 80 characters? It is the Hadoop code style.
          Tsz Wo Nicholas Sze made changes -
          Assignee Kay Kay [ kaykay.unique ]
          Priority Major [ 3 ] Minor [ 4 ]
          Component/s documentation [ 12312918 ]
          Karthik K made changes -
          Attachment HDFS-930.patch [ 12435375 ]
          Karthik K made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Karthik K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Karthik K added a comment -

          Could you reformat the new codes so that each line only takes <= 80 characters? It is the Hadoop code style.

          Done.

          Show
          Karthik K added a comment - Could you reformat the new codes so that each line only takes <= 80 characters? It is the Hadoop code style. Done.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good. Thanks, Kay Kay.

          This only changes an error message. So, no new test is needed.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Thanks, Kay Kay. This only changes an error message. So, no new test is needed.
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags [Reviewed]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The previous hudson report page cannot be loaded. Not sure which tests were failed. Nonetheless, I don't think the failed tests are related to this. Anyway, try submitting again.

          Show
          Tsz Wo Nicholas Sze added a comment - The previous hudson report page cannot be loaded. Not sure which tests were failed. Nonetheless, I don't think the failed tests are related to this. Anyway, try submitting again.
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Karthik K added a comment -

          I don't think the failed tests are related to this. Anyway, try submitting again.

          Could access the failure log. Not sure - if that is related to this either. Had canceled and resubmitted the patch again to be picked up by Hudson. Might need manual intervention if hudson fails for unrelated reason.

          Show
          Karthik K added a comment - I don't think the failed tests are related to this. Anyway, try submitting again. Could access the failure log. Not sure - if that is related to this either. Had canceled and resubmitted the patch again to be picked up by Hudson. Might need manual intervention if hudson fails for unrelated reason.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12435375/HDFS-930.patch
          against trunk revision 907964.

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435375/HDFS-930.patch against trunk revision 907964. +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/225/console This message is automatically generated.
          Hide
          Karthik K added a comment -

          Failed 1 test:
          ---------------

          org.apache.hadoop.hdfs.security.TestDelegationToken.testDelegationTokenWithRealUser (from TestDelegationToken)

          Error message for the failure is :

          User: RealUser is not allowed to impersonate proxyUser
          Stacktrace

          org.apache.hadoop.ipc.RemoteException: User: RealUser is not allowed to impersonate proxyUser
          at org.apache.hadoop.ipc.Client.call(Client.java:860)
          at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:198)
          at $Proxy7.getProtocolVersion(Unknown Source)
          at org.apache.hadoop.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:228)
          at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:220)
          at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:151)
          at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:250)
          at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:217)
          at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:87)
          at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1747)
          at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:69)
          at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1775)
          at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1763)
          at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:193)
          at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:101)
          at org.apache.hadoop.hdfs.MiniDFSCluster.getFileSystem(MiniDFSCluster.java:813)
          at org.apache.hadoop.hdfs.security.TestDelegationToken$1.run(TestDelegationToken.java:147)
          at org.apache.hadoop.hdfs.security.TestDelegationToken$1.run(TestDelegationToken.java:145)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:396)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:650)
          at org.apache.hadoop.hdfs.security.TestDelegationToken.testDelegationTokenWithRealUser(TestDelegationToken.java:144)

          Show
          Karthik K added a comment - Failed 1 test: --------------- org.apache.hadoop.hdfs.security.TestDelegationToken.testDelegationTokenWithRealUser (from TestDelegationToken) Error message for the failure is : User: RealUser is not allowed to impersonate proxyUser Stacktrace org.apache.hadoop.ipc.RemoteException: User: RealUser is not allowed to impersonate proxyUser at org.apache.hadoop.ipc.Client.call(Client.java:860) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:198) at $Proxy7.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:228) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:220) at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:151) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:250) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:217) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:87) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1747) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:69) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1775) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1763) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:193) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:101) at org.apache.hadoop.hdfs.MiniDFSCluster.getFileSystem(MiniDFSCluster.java:813) at org.apache.hadoop.hdfs.security.TestDelegationToken$1.run(TestDelegationToken.java:147) at org.apache.hadoop.hdfs.security.TestDelegationToken$1.run(TestDelegationToken.java:145) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:650) at org.apache.hadoop.hdfs.security.TestDelegationToken.testDelegationTokenWithRealUser(TestDelegationToken.java:144)
          Hide
          Konstantin Shvachko added a comment -

          This is HDFS-965.

          Show
          Konstantin Shvachko added a comment - This is HDFS-965 .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The failed test is not related to this.

          I have committed this. Thanks, Kay Kay!

          Show
          Tsz Wo Nicholas Sze added a comment - The failed test is not related to this. I have committed this. Thanks, Kay Kay!
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/)

          Show
          Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/ )
          Tom White made changes -
          Fix Version/s 0.21.0 [ 12314046 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Karthik K
              Reporter:
              Karthik K
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development