[HBASE-8394] Utilize isFileClosed() so that the wait for lease recovery can be optimized

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 23/Apr/13 03:20
• Updated: 16/Jun/22 06:01
• Resolved: 23/Apr/13 05:20

Description

In HDFS-4525, the following API was added:

  /**
   * Get the close status of a file
   * @param src The path to the file
   *
   * @return return true if file is closed
   * @throws FileNotFoundException if the file does not exist.
   * @throws IOException If an I/O error occurred     
   */
  public boolean isFileClosed(Path src) throws IOException {

It is not available in hadoop 1.1 at the moment.

In HBASE-8389, there was discussion for utilizing isFileClosed() to optimize the wait time for lease recovery.

Subsequent lease recovery request for the same Path would preempt outstanding recovery. But if we wait for one particular recovery request to complete without limit on duration, we may incur penalty due to various failure scenarios such as the primary Data Node chosen by Namenode being stale, etc.

This JIRA continues the work for 0.95 / trunk.

Issue Links

• is related to: HDFS-4525 Provide an API for knowing that whether file is closed or not.
• is related to: HBASE-8389 HBASE-8354 forces Namenode into loop with lease recovery requests
• is related to: HBASE-8449 Refactor recoverLease retries and pauses informed by findings over in hbase-8389