Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1605

Convert DFSInputStream synchronized sections to a ReadWrite lock

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:

      Description

      Hbase does concurrent preads from multiple threads to different blocks of the same hdfs file. Each of these pread calls invoke DFSInputStream.getFileLength() and DFSInputStream.getBlockAt(). These methods are "synchronized", thus causing all the concurrent threads to serialize. It would help performance to convert this to a Read/Write lock

        Attachments

        1. DFSClientRWlock.1.txt
          11 kB
          Dhruba Borthakur
        2. DFSClientRWlock.3.txt
          11 kB
          Dhruba Borthakur
        3. HADOOP-1605-trunk.patch
          45 kB
          kartheek muthyala
        4. HADOOP-1605-trunk-1.patch
          80 kB
          kartheek muthyala
        5. HADOOP-1605-trunk-2.patch
          46 kB
          kartheek muthyala
        6. HDFS-1605.txt
          45 kB
          Liang Xie

          Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1599 Umbrella Jira for Improving HBASE support in HDFS

          • Major - Major loss of function.
          • Open

            Activity

              People

              • Assignee:
                kartheek kartheek muthyala
                Reporter:
                dhruba Dhruba Borthakur
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated: