Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-505 OzoneManager HA
  3. HDDS-1892

HA failover attempt log level should be set to DEBUG

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 1.0.0
    • Component/s: OM HA, Ozone Manager
    • Labels:
      None

      Description

      Current HA failover log level is INFO:

      $ ozone sh volume create /volume
      2019-08-01 20:55:03 INFO RetryInvocationHandler:411 — com.google.protobuf.ServiceException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ozone.om.exceptions.NotLeader Exception): OM om2 is not the leader. Could not determine the leader node.
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.createNotleaderException(OzoneManagerProtocolServerSideTranslatorPB.java:183)
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:172)
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:94)
              at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
              at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
              at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
      , while invoking $Proxy19.submitRequest over nodeld=om2,nodeAddress=om2:9862 after 1 failover attempts. Trying to failover immediately.
      2019-08-01 20:55:03 INFO RpcClient:288 — Creating Volume: volume, with hadoop as owner.
      VOLUME_ALREADY_EXISTS Volume already exists
      
      $ ozone sh bucket create /volume/bucket
      2019-08-01 20:55:23 INFO RetryInvocationHandler:411 — com.google.protobuf.ServiceException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ozone.om.exceptions.Notleader Exception): OM om2 is not the leader. Could not determine the leader node.
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.createNotLeaderException(OzoneManagerProtocolServerSideTranslatorPB.java:183)
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:172)
              at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:94)
              at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
              at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
              at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
      , while invoking $Proxy19.submitRequest over nodeId.om2,nodeAddress.om2:9862 after 1 failover attempts. Trying to failover immediately.
      2019-08-01 20:55:24 INFO RpcClient:425 — Creating Bucket: volume/bucket, with Versioning false and Storage Type set to DISK and Encryption set to false
      

      The client will print out every failover attempt.

      Suppress it by setting the log level to DEBUG.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hanishakoneru Hanisha Koneru
                Reporter:
                smeng Siyao Meng
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: