Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13762 Support non-volatile storage class memory(SCM) in HDFS cache directives
  3. HDFS-14402

Use FileChannel.transferTo() method for transferring block to SCM cache

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0
    • caching, datanode

    Description

      We will consider to use transferTo API to improve SCM's cach performace.

      Discussion: 5th comment in HDFS-14355

      Comment: FileMappableBlockLoader : Currently you are reading data from input stream and verifying checksum and writing that buffer to MBB. One thought here is: how about using FileChannel#transferTo API for transferring data from one channel to other natively. and then do mmap on destination file(assuming mmap may be faster in target file) and do checksum verification on it? 

      Attachments

        1. HDFS-14402.000.patch
          13 kB
          Feilong He
        2. HDFS-14402.001.patch
          13 kB
          Feilong He
        3. HDFS-14402.002.patch
          14 kB
          Feilong He
        4. With-Cache-Improvement-Patch.png
          65 kB
          Feilong He
        5. Without-Cache-Improvement-Patch.png
          73 kB
          Feilong He

        Issue Links

          relates to

          Sub-task - The sub-task of the issue HDFS-14355 Implement HDFS cache on SCM by using pure java mapped byte buffer

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              PhiloHe Feilong He
              PhiloHe Feilong He
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: