Description
HBase relies on FileSystems implementing CanUnbuffer.unbuffer() to force input streams to free up remote connections (HBASE-9393). This works for HDFS, but not elsewhere.
S3A input stream can implement CanUnbuffer.unbuffer() by closing the input stream and relying on lazy seek to reopen it on demand.
Needs
- Contract specification of unbuffer. As in "who added a new feature to filesystems but forgot to mention what it should do?"
- Contract test for filesystems which declare their support.
- S3AInputStream to call closeStream() on a call to unbuffer().
- Test case
Attachments
Issue Links
- is related to
-
HADOOP-14748 Wasb input streams to implement CanUnbuffer
-
- Resolved
-
-
HADOOP-14872 CryptoInputStream should implement unbuffer
-
- Resolved
-
-
HADOOP-12805 Annotate CanUnbuffer with @InterfaceAudience.Public
-
- Closed
-
- relates to
-
HADOOP-14765 AdlFsInputStream should implement unbuffer
-
- Resolved
-
-
HBASE-9393 Region Server fails to properly close socket resulting in many CLOSE_WAIT to Data Nodes
-
- Closed
-
-
HDFS-7694 FSDataInputStream should support "unbuffer"
-
- Closed
-
- links to