Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-941

Datanode xceiver protocol should allow reuse of a connection

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Right now each connection into the datanode xceiver only processes one operation.

      In the case that an operation leaves the stream in a well-defined state (eg a client reads to the end of a block successfully) the same connection could be reused for a second operation. This should improve random read performance significantly.

      Attachments

        1. HDFS-941-6.patch
          55 kB
          bc Wong
        2. HDFS-941-6.patch
          55 kB
          bc Wong
        3. HDFS-941-6.patch
          55 kB
          bc Wong
        4. HDFS-941-6.22.patch
          55 kB
          Michael Stack
        5. HDFS-941-5.patch
          82 kB
          bc Wong
        6. HDFS-941-4.patch
          64 kB
          bc Wong
        7. HDFS-941-3.patch
          63 kB
          bc Wong
        8. HDFS-941-3.patch
          63 kB
          bc Wong
        9. HDFS-941-2.patch
          62 kB
          bc Wong
        10. hdfs941-1.png
          7 kB
          Kihwal Lee
        11. HDFS-941-1.patch
          42 kB
          bc Wong
        12. hdfs-941.txt
          56 kB
          Todd Lipcon
        13. hdfs-941.txt
          58 kB
          Todd Lipcon
        14. hdfs-941.txt
          60 kB
          Todd Lipcon
        15. hdfs-941.txt
          60 kB
          Todd Lipcon
        16. fix-close-delta.txt
          4 kB
          Todd Lipcon
        17. 941.22.v3.txt
          61 kB
          Michael Stack
        18. 941.22.v2.txt
          63 kB
          Michael Stack
        19. 941.22.txt
          63 kB
          Michael Stack
        20. 941.22.txt
          63 kB
          Michael Stack

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-1001 DataXceiver and BlockReader disagree on when to send/recv CHECKSUM_OK

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3359 DFSClient.close should close cached sockets

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1599 Umbrella Jira for Improving HBASE support in HDFS

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-3529 Add search to HBase

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-2071 Use of isConnected() in DataXceiver is invalid

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-380 support for persistent connections to improve random read performance.

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1148 Convert FSDataset to ReadWriteLock

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              bcwalrus bc Wong
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: