[HADOOP-15051] FSDataOutputStream returned by LocalFileSystem#createNonRecursive doesn't have hflush capability

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.0.0-beta1
• Component/s: fs
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 17/Nov/17 21:08
• Updated: 09/Jan/19 21:57
• Resolved: 19/Nov/17 11:08

Description

See HBASE-19289 for background information.
Here is related hbase code (fs is instance of LocalFileSystem):

    this.output = fs.createNonRecursive(path, overwritable, bufferSize, replication, blockSize,
      null);
    // TODO Be sure to add a check for hsync if this branch includes HBASE-19024
    if (!(CommonFSUtils.hasCapability(output, "hflush"))) {
      throw new StreamLacksCapabilityException("hflush");

StreamCapabilities is used to poll "hflush" capability.

busbey suggested fixing this in hadoop.

Issue Links

• is related to: HADOOP-7844 Flush/sync do not work on Hadoop LocalFileSystem
• is related to: HBASE-19289 CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
• relates to: HADOOP-13327 Add OutputStream + Syncable to the Filesystem Specification