Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13610

[Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      See HDFS-13150 for full design.

      This JIRA is targeted at cleanup tasks:

      • Add in integration testing. We can expand TestStandbyInProgressTail
      • Documentation in HDFSHighAvailabilityWithQJM
      • Remove the dummy sync added as part of HDFS-10519; it is unnecessary since now in-progress tailing does not rely on the JN committedTxnId

      A few bugs are also fixed:

      • Due to changes in HDFS-13609 to enable use of the RPC mechanism whenever inProgressOK is true, there were codepaths which would use the RPC mechanism even when dfs.ha.tail-edits.in-progress was false, meaning that the JNs did not enable the cache. Update the QJM logic to only use selectRpcInputStreams if this config is true.
      • Fix a false error logged when the layout version changes
      • Fix the logging when a layout version change occurs to avoid printing out a placeholder value (Integer.MAX_VALUE)

        Attachments

        1. HDFS-13610-HDFS-12943.004.patch
          31 kB
          Erik Krogen
        2. HDFS-13610-HDFS-12943.003.patch
          25 kB
          Erik Krogen
        3. HDFS-13610-HDFS-12943.002.patch
          24 kB
          Erik Krogen
        4. HDFS-13610-HDFS-12943.001.patch
          15 kB
          Erik Krogen
        5. HDFS-13610-HDFS-12943.000.patch
          13 kB
          Erik Krogen

          Issue Links

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: