Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11914

Add more diagnosis info for fsimage transfer failure.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha4
    • None
    • Reviewed

    Description

      Hit a fsimage download problem:

      Client tries to download fsimage, and got:

      WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: File http://x.y.z:50070/imagetransfer?getimage=1&txid=latest received length xyz is not of the advertised size abc.

      Basically client does not get enough fsimage data and finished prematurely without any exception thrown, until it finds the size of data received is smaller than expected. The client then closed the conenction to NN, that caused NN to report

      INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Connection closed by client

      This jira is to add some more information in logs to help debugging the sitaution. Specifically, report the stack trace when the connection is closed. And how much data has been sent at that point. etc.

      Attachments

        1. HDFS-11914.001.patch
          3 kB
          Yongjun Zhang
        2. HDFS-11914.002.patch
          3 kB
          Yongjun Zhang
        3. HDFS-11914.003.patch
          3 kB
          Yongjun Zhang
        4. HDFS-11914.004.patch
          4 kB
          Yongjun Zhang
        5. HDFS-11914.branch-2.001.patch
          3 kB
          Yongjun Zhang

        Activity

          People

            yzhangal Yongjun Zhang
            yzhangal Yongjun Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: