[HDFS-4157] libhdfs: hdfsTell could be implemented a smarter than it is

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.0.3-alpha
• Component/s: libhdfs
• Target Version/s: 2.0.3-alpha
• Assignee: Unassigned
• Reporter: Colin McCabe
• Created: 06/Nov/12 23:03
• Updated: 03/Dec/12 23:42

Description

In libhdfs, hdfsTell currently makes a JNI call into DFSClient to determine the position of the stream. However, we could cache this information easily, since libhdfs controls access to the stream. This would avoid the overhead of JNI.

This would be very helpful for fuse_dfs, since that program calls hdfsTell before every write or read operation. This can be quite a lot of overhead, since writes may be as small as 4kb (depends on FUSE configuration, kernel version, etc.)