Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14942

Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.3.0, 3.1.4, 3.2.2
    • None
    • None
    • Reviewed

    Description

      when hadoop 2.x upgrades to hadoop 3.x,  InterQJournalProtocol is newly added,so  throw Unknown protocol. 

      the newly InterQJournalProtocol is used to sychronize past log segments to JNs that missed them.  And that an error occurs does not affect normal service. I think it should not be a ERROR log,and that log a warn log is more reasonable.

       private void syncWithJournalAtIndex(int index) {
        ...
          GetEditLogManifestResponseProto editLogManifest;
          try {
            editLogManifest = jnProxy.getEditLogManifestFromJournal(jid,
                nameServiceId, 0, false);
          } catch (IOException e) {
            LOG.error("Could not sync with Journal at " +
                otherJNProxies.get(journalNodeIndexForSync), e);
            return;
          }
      
      2019-10-30,15:11:17,388 ERROR org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer: Could not sync with Journal at mos1-hadoop-prc-ct17.ksru/10.85.3.59:111002019-10-30,15:11:17,388 ERROR org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer: Could not sync with Journal at mos1-hadoop-prc-ct17.ksru/10.85.3.59:11100org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unknown protocol: org.apache.hadoop.hdfs.qjournal.protocol.InterQJournalProtocol
      at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1565)
      at org.apache.hadoop.ipc.Client.call(Client.java:1511)
      at org.apache.hadoop.ipc.Client.call(Client.java:1421)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
      at com.sun.proxy.$Proxy16.getEditLogManifestFromJournal(Unknown Source)
      at org.apache.hadoop.hdfs.qjournal.protocolPB.InterQJournalProtocolTranslatorPB.getEditLogManifestFromJournal(InterQJournalProtocolTranslatorPB.java:75)
      at org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer.syncWithJournalAtIndex(JournalNodeSyncer.java:250)
      at org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer.syncJournals(JournalNodeSyncer.java:226)
      at org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer.lambda$startSyncJournalsDaemon$0(JournalNodeSyncer.java:186)
      at java.lang.Thread.run(Thread.java:748)
      

      Attachments

        1. HDFS-14942.001.patch
          1 kB
          Lisheng Sun
        2. HDFS-14942.002.patch
          1 kB
          Lisheng Sun
        3. HDFS-14942.003.patch
          1 kB
          Lisheng Sun
        4. HDFS-14942.004.patch
          1 kB
          Lisheng Sun

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            leosun08 Lisheng Sun
            leosun08 Lisheng Sun
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment