Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11292

log lastWrittenTxId etc info in logSyncAll

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For the issue reported in HDFS-10943, even after HDFS-7964's fix is included, the problem still exists, this means there might be some synchronization issue.

      To diagnose that, create this jira to report the lastWrittenTxId info in logSyncAll() call, such that we can compare against the error message reported in HDFS-7964

      Specifically, there is two possibility for the HDFS-10943 issue:

      1. logSyncAll() (statement A in the code quoted below) doesn't flush all requested txs for some reason

      2. logSyncAll() does flush all requested txs, but some new txs sneaked in between A and B. It's observed that the lastWrittenTxId in B and C are the same.

      This proposed reporting would help confirming if 2 is true.

       public synchronized void endCurrentLogSegment(boolean writeEndTxn) {
          LOG.info("Ending log segment " + curSegmentTxId);
          Preconditions.checkState(isSegmentOpen(),
              "Bad state: %s", state);
      
          if (writeEndTxn) {
            logEdit(LogSegmentOp.getInstance(cache.get(),
                FSEditLogOpCodes.OP_END_LOG_SEGMENT));
          }
          // always sync to ensure all edits are flushed.
      A.    logSyncAll();
      
      B.    printStatistics(true);
      
          final long lastTxId = getLastWrittenTxId();
      
          try {
      C.      journalSet.finalizeLogSegment(curSegmentTxId, lastTxId);
            editLogStream = null;
          } catch (IOException e) {
            //All journals have failed, it will be handled in logSync.
          }
      
          state = State.BETWEEN_LOG_SEGMENTS;
        }
      

        Attachments

        1. HDFS-11292.003.patch
          2 kB
          Yongjun Zhang
        2. HDFS-11292.002.patch
          2 kB
          Yongjun Zhang
        3. HDFS-11292.001.patch
          2 kB
          Yongjun Zhang

          Issue Links

            Activity

              People

              • Assignee:
                yzhangal Yongjun Zhang
                Reporter:
                yzhangal Yongjun Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: