Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11914

Add more diagnosis info for fsimage transfer failure.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Hit a fsimage download problem:

      Client tries to download fsimage, and got:

      WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: File http://x.y.z:50070/imagetransfer?getimage=1&txid=latest received length xyz is not of the advertised size abc.

      Basically client does not get enough fsimage data and finished prematurely without any exception thrown, until it finds the size of data received is smaller than expected. The client then closed the conenction to NN, that caused NN to report

      INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Connection closed by client

      This jira is to add some more information in logs to help debugging the sitaution. Specifically, report the stack trace when the connection is closed. And how much data has been sent at that point. etc.

        Attachments

        1. HDFS-11914.branch-2.001.patch
          3 kB
          Yongjun Zhang
        2. HDFS-11914.004.patch
          4 kB
          Yongjun Zhang
        3. HDFS-11914.003.patch
          3 kB
          Yongjun Zhang
        4. HDFS-11914.002.patch
          3 kB
          Yongjun Zhang
        5. HDFS-11914.001.patch
          3 kB
          Yongjun Zhang

          Activity

            People

            • Assignee:
              yzhangal Yongjun Zhang
              Reporter:
              yzhangal Yongjun Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: