Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2296

If read error while lease is being recovered, client reverts to stale view on block info

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 0.20-append, 0.22.0, 0.23.0
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      We are seeing the following issue around recoverLease over in hbaselandia. DFSClient calls recoverLease to assume ownership of a file. The recoverLease returns to the client but it can take time for the new state to propagate. Meantime, an incoming read fails though its using updated block info. Thereafter all read retries fail because on exception we revert to stale block view and we never recover. Laxman reports this issue in the below mailing thread:

      See this thread for first report of this issue: http://search-hadoop.com/m/S1mOHFRmgk2/%2527FW%253A+Handling+read+failures+during+recovery%2527&subj=FW+Handling+read+failures+during+recovery

      Chatting w/ Hairong offline, she suggests this a general issue around lease recovery no matter how it triggered (new recoverLease or not).

      I marked this critical. At least over in hbase it is since we get set stuck here recovering a crashed server.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated: