[HDFS-2129] Simplify BlockReader to not inherit from FSInputChecker - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.23.1
Component/s: hdfs-client, performance
Labels:
None

Hadoop Flags:

Reviewed
Release Note:
BlockReader has been reimplemented to use direct byte buffers. If you use a custom socket factory, it must generate sockets that have associated Channels.

Description

BlockReader is currently quite complicated since it has to conform to the FSInputChecker inheritance structure. It would be much simpler to implement it standalone. Benchmarking indicates it's slightly faster, as well.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

seq-read-1gb-bench.png
05/Aug/11 18:31
39 kB
Todd Lipcon
hdfs-2129-benchmark.png
05/Aug/11 17:57
124 kB
Todd Lipcon
hdfs-2129.txt
14/Jul/11 21:02
32 kB
Todd Lipcon
hdfs-2129.txt
05/Aug/11 17:57
33 kB
Todd Lipcon
hdfs-2129.txt
18/Aug/11 03:54
44 kB
Todd Lipcon
hdfs-2129.txt
24/Oct/11 19:15
48 kB
Todd Lipcon
hdfs-2129.txt
28/Oct/11 22:31
52 kB
Todd Lipcon
hdfs-2129.txt
02/Nov/11 20:08
63 kB
Todd Lipcon
hdfs-2129.txt
02/Nov/11 21:04
63 kB
Todd Lipcon
hdfs-2129.txt
03/Nov/11 00:02
63 kB
Todd Lipcon

Issue Links

is blocked by

HADOOP-7444 Add Checksum API to verify and calculate checksums "in bulk"

Closed

HDFS-2260 Refactor BlockReader into an interface and implementation

Closed

Activity

People

Assignee:: Todd Lipcon

Reporter:: Todd Lipcon

Votes:: 0 Vote for this issue

Watchers:: 9 Start watching this issue

Dates

Created:: 06/Jul/11 04:32

Updated:: 10/Mar/15 02:12

Resolved:: 03/Nov/11 06:56