Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13689

NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode



    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hdfs, namenode
    • Labels:
    • Target Version/s:


      NameNodeRpcServer#getEditsFromTxid currently decides which transactions are able to be served, i.e. which transactions are durable, using the following logic:

          long syncTxid = log.getSyncTxId();
          // If we haven't synced anything yet, we can only read finalized
          // segments since we can't reliably determine which txns in in-progress
          // segments have actually been committed (e.g. written to a quorum of JNs).
          // If we have synced txns, we can definitely read up to syncTxid since
          // syncTxid is only updated after a transaction is committed to all
          // journals. (In-progress segments written by old writers are already
          // discarded for us, so if we read any in-progress segments they are
          // guaranteed to have been written by this NameNode.)
          boolean readInProgress = syncTxid > 0;

      This assumes that the NameNode serving this request is the current writer/active NameNode, which may not be true in the ObserverNode situation. Since selectInputStreams now has a onlyDurableTxns flag, which, if enabled, will only return durable/committed transactions, we can instead leverage this to provide the same functionality. We should utilize this to avoid consistency issues when serving this request from the ObserverNode.




            • Assignee:
              xkrogen Erik Krogen
            • Votes:
              0 Vote for this issue
              3 Start watching this issue


              • Created: