Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4157

libhdfs: hdfsTell could be implemented a smarter than it is

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.0.3-alpha
    • Fix Version/s: None
    • Component/s: libhdfs
    • Labels:
      None

      Description

      In libhdfs, hdfsTell currently makes a JNI call into DFSClient to determine the position of the stream. However, we could cache this information easily, since libhdfs controls access to the stream. This would avoid the overhead of JNI.

      This would be very helpful for fuse_dfs, since that program calls hdfsTell before every write or read operation. This can be quite a lot of overhead, since writes may be as small as 4kb (depends on FUSE configuration, kernel version, etc.)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              cmccabe Colin McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: