Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10429

DataStreamer interrupted warning always appears when using CLI upload file

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 2.7.3
    • None
    • None
    • None

    Description

      Every time I use 'hdfs dfs -put' upload file, this warning is printed:

      16/05/18 20:57:56 WARN hdfs.DataStreamer: Caught exception
      java.lang.InterruptedException
      	at java.lang.Object.wait(Native Method)
      	at java.lang.Thread.join(Thread.java:1245)
      	at java.lang.Thread.join(Thread.java:1319)
      	at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:871)
      	at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:519)
      	at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:696)
      

      The reason is this: originally, DataStreamer::closeResponder always prints a warning about InterruptedException; since HDFS-9812, DFSOutputStream::closeImpl always forces threads to close, which causes InterruptedException.

      A simple fix is to use debug level log instead of warning level.

      Attachments

        1. HDFS-10429.1.patch
          0.7 kB
          Zhiyuan Yang
        2. HDFS-10429.2.patch
          0.8 kB
          Zhiyuan Yang
        3. HDFS-10429.3.patch
          2 kB
          Wei-Chiu Chuang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            zhiyuany Zhiyuan Yang

            Dates

              Created:
              Updated:

              Slack

                Issue deployment