[HADOOP-12994] Specify PositionedReadable, add contract tests, fix problems

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: fs
• Hadoop Flags: Reviewed
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 03/Apr/16 15:24
• Updated: 21/Jan/21 13:43
• Resolved: 08/Apr/16 20:54

Description

Some work on S3a has shown up that there aren't tests catching regressions in readFully, reviewing the documentation shows that its specification could be improved.

1. review the spec
2. review the implementations
3. add tests (proposed: to the seek contract; streams which support seek should support positioned readable)
4. fix code, where it differs significantly from HDFS or LocalFS

Issue Links

• breaks: HDFS-10254 DfsClient undervalidates args for PositionedReadable operations
• breaks: HDFS-10277 PositionedReadable test testReadFullyZeroByteFile failing in HDFS
• breaks: HDFS-10291 TestShortCircuitLocalRead failing
• contains: HADOOP-12997 s3a to pass PositionedReadable contract tests, improve readFully perf.
• contains: HDFS-10254 DfsClient undervalidates args for PositionedReadable operations
• contains: HDFS-10255 ByteRangeInputStream.readFully leaks stream handles on failure
• incorporates: HADOOP-12997 s3a to pass PositionedReadable contract tests, improve readFully perf.
• incorporates: HADOOP-12996 remove @InterfaceAudience.LimitedPrivate({"HDFS"}) from FSInputStream
• is depended upon by: HADOOP-12444 Support lazy seek in S3AInputStream
• is depended upon by: HADOOP-12997 s3a to pass PositionedReadable contract tests, improve readFully perf.
• is related to: HADOOP-13559 Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes
• is related to: HADOOP-13017 Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0
• is related to: HADOOP-9361 Strictly define the expected behavior of filesystem APIs and write tests to verify compliance