Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13689

NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • hdfs, namenode
    • None

    Description

      NameNodeRpcServer#getEditsFromTxid currently decides which transactions are able to be served, i.e. which transactions are durable, using the following logic:

          long syncTxid = log.getSyncTxId();
          // If we haven't synced anything yet, we can only read finalized
          // segments since we can't reliably determine which txns in in-progress
          // segments have actually been committed (e.g. written to a quorum of JNs).
          // If we have synced txns, we can definitely read up to syncTxid since
          // syncTxid is only updated after a transaction is committed to all
          // journals. (In-progress segments written by old writers are already
          // discarded for us, so if we read any in-progress segments they are
          // guaranteed to have been written by this NameNode.)
          boolean readInProgress = syncTxid > 0;
      

      This assumes that the NameNode serving this request is the current writer/active NameNode, which may not be true in the ObserverNode situation. Since selectInputStreams now has a onlyDurableTxns flag, which, if enabled, will only return durable/committed transactions, we can instead leverage this to provide the same functionality. We should utilize this to avoid consistency issues when serving this request from the ObserverNode.

      Attachments

        Activity

          People

            Unassigned Unassigned
            xkrogen Erik Krogen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: