Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Distcp with hftp is failing.

      $hadoop   distcp hftp://<NNhostname>:50070/user/hadoopqa/1316814737/newtemp 1316814737/as
      11/09/23 21:52:33 INFO tools.DistCp: srcPaths=[hftp://<NNhostname>:50070/user/hadoopqa/1316814737/newtemp]
      11/09/23 21:52:33 INFO tools.DistCp: destPath=1316814737/as
      Retrieving token from: https://<NN IP>:50470/getDelegationToken
      Retrieving token from: https://<NN IP>:50470/getDelegationToken?renewer=mapred
      11/09/23 21:52:34 INFO security.TokenCache: Got dt for hftp://<NNhostname>:50070/user/hadoopqa/1316814737/newtemp;uri=<NN IP>:50470;t.service=<NN IP>:50470
      org.apache.hadoop.ipc.RemoteException: name=hadoopqa != expected=null
      	at org.apache.hadoop.ipc.RemoteException.valueOf(RemoteException.java:102)
      	at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:385)
      	...
      
      1. HDFS-2361-20s.1.patch
        1 kB
        Jitendra Nath Pandey
      2. HDFS-2361-20s.2.patch
        1 kB
        Jitendra Nath Pandey
      3. HDFS-2361-20s.3.patch
        2 kB
        Jitendra Nath Pandey
      4. HDFS-2361.trunk.1.patch
        3 kB
        Jitendra Nath Pandey
      5. HDFS-2361.trunk.2.patch
        3 kB
        Jitendra Nath Pandey

        Issue Links

          Activity

          Allen Wittenauer made changes -
          Fix Version/s 2.0.0-alpha [ 12320353 ]
          Allen Wittenauer made changes -
          Fix Version/s 2.0.0-alpha [ 12320353 ]
          Fix Version/s 0.24.0 [ 12317653 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-2369 [ HDFS-2369 ]
          Gavin made changes -
          Link This issue blocks HDFS-2369 [ HDFS-2369 ]
          Suresh Srinivas made changes -
          Fix Version/s 1.1.0 [ 12317959 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to HDFS-2316 [ HDFS-2316 ]
          Tsz Wo Nicholas Sze made changes -
          Fix Version/s 0.23.0 [ 12315571 ]
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Daryn Sharp made changes -
          Link This issue relates to HDFS-2380 [ HDFS-2380 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.20.205.0 [ 12316392 ]
          Fix Version/s 0.20.206.0 [ 12317959 ]
          Fix Version/s 0.24.0 [ 12317653 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Description Distcp with hftp is failing.


          $hadoop distcp hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp 1316814737/as
          11/09/23 21:52:33 INFO tools.DistCp: srcPaths=[hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp]
          11/09/23 21:52:33 INFO tools.DistCp: destPath=1316814737/as
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken?renewer=mapred
          11/09/23 21:52:34 INFO security.TokenCache: Got dt for hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp;uri=&lt;NN IP>:50470;t.service=<NN IP>:50470
          org.apache.hadoop.ipc.RemoteException: name=hadoopqa != expected=null
          at org.apache.hadoop.ipc.RemoteException.valueOf(RemoteException.java:102)
          at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:385)
          at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
          at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
          at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
          at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:626)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3103)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:922)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
          at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
          at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
          at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
          at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:426)
          at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:437)
          at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:466)
          at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:708)
          at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:636)
          at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
          at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
          at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
          Distcp with hftp is failing.

          {noformat}
          $hadoop distcp hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp 1316814737/as
          11/09/23 21:52:33 INFO tools.DistCp: srcPaths=[hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp]
          11/09/23 21:52:33 INFO tools.DistCp: destPath=1316814737/as
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken?renewer=mapred
          11/09/23 21:52:34 INFO security.TokenCache: Got dt for hftp://&lt;NNhostname&gt;:50070/user/hadoopqa/1316814737/newtemp;uri=&lt;NN IP>:50470;t.service=<NN IP>:50470
          org.apache.hadoop.ipc.RemoteException: name=hadoopqa != expected=null
          at org.apache.hadoop.ipc.RemoteException.valueOf(RemoteException.java:102)
          at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:385)
          ...
          {noformat}
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-2361.trunk.2.patch [ 12496649 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags Reviewed [ 10343 ]
          Jitendra Nath Pandey made changes -
          Link This issue blocks HDFS-2369 [ HDFS-2369 ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-2361.trunk.1.patch [ 12496624 ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-2361-20s.3.patch [ 12496619 ]
          Jitendra Nath Pandey made changes -
          Assignee Jitendra Nath Pandey [ jnp ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-2361-20s.2.patch [ 12496593 ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-2361-20s.1.patch [ 12496592 ]
          Rajit Saha made changes -
          Summary distcp with hftp is failing hftp is broken
          Rajit Saha made changes -
          Release Note It seems hftp itself is broken


          $hadoop dfs -lsr hftp://&lt;NN hostname>:50070/tmp
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken
          lsr: name=hadoopqa != expected=null
          Rajit Saha made changes -
          Field Original Value New Value
          Release Note It seems hftp itself is broken


          $hadoop dfs -lsr hftp://&lt;NN hostname>:50070/tmp
          Retrieving token from: https://&lt;NN IP>:50470/getDelegationToken
          lsr: name=hadoopqa != expected=null
          Priority Major [ 3 ] Critical [ 2 ]
          Rajit Saha created issue -

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Rajit Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development