Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13689

NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode



    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • hdfs, namenode
    • None


      NameNodeRpcServer#getEditsFromTxid currently decides which transactions are able to be served, i.e. which transactions are durable, using the following logic:

          long syncTxid = log.getSyncTxId();
          // If we haven't synced anything yet, we can only read finalized
          // segments since we can't reliably determine which txns in in-progress
          // segments have actually been committed (e.g. written to a quorum of JNs).
          // If we have synced txns, we can definitely read up to syncTxid since
          // syncTxid is only updated after a transaction is committed to all
          // journals. (In-progress segments written by old writers are already
          // discarded for us, so if we read any in-progress segments they are
          // guaranteed to have been written by this NameNode.)
          boolean readInProgress = syncTxid > 0;

      This assumes that the NameNode serving this request is the current writer/active NameNode, which may not be true in the ObserverNode situation. Since selectInputStreams now has a onlyDurableTxns flag, which, if enabled, will only return durable/committed transactions, we can instead leverage this to provide the same functionality. We should utilize this to avoid consistency issues when serving this request from the ObserverNode.




            Unassigned Unassigned
            xkrogen Erik Krogen
            0 Vote for this issue
            4 Start watching this issue