Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-941

Datanode xceiver protocol should allow reuse of a connection

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Right now each connection into the datanode xceiver only processes one operation.

      In the case that an operation leaves the stream in a well-defined state (eg a client reads to the end of a block successfully) the same connection could be reused for a second operation. This should improve random read performance significantly.

      Attachments

        1. 941.22.txt
          63 kB
          Michael Stack
        2. 941.22.txt
          63 kB
          Michael Stack
        3. 941.22.v2.txt
          63 kB
          Michael Stack
        4. 941.22.v3.txt
          61 kB
          Michael Stack
        5. fix-close-delta.txt
          4 kB
          Todd Lipcon
        6. hdfs-941.txt
          60 kB
          Todd Lipcon
        7. hdfs-941.txt
          60 kB
          Todd Lipcon
        8. hdfs-941.txt
          58 kB
          Todd Lipcon
        9. hdfs-941.txt
          56 kB
          Todd Lipcon
        10. HDFS-941-1.patch
          42 kB
          bc Wong
        11. hdfs941-1.png
          7 kB
          Kihwal Lee
        12. HDFS-941-2.patch
          62 kB
          bc Wong
        13. HDFS-941-3.patch
          63 kB
          bc Wong
        14. HDFS-941-3.patch
          63 kB
          bc Wong
        15. HDFS-941-4.patch
          64 kB
          bc Wong
        16. HDFS-941-5.patch
          82 kB
          bc Wong
        17. HDFS-941-6.22.patch
          55 kB
          Michael Stack
        18. HDFS-941-6.patch
          55 kB
          bc Wong
        19. HDFS-941-6.patch
          55 kB
          bc Wong
        20. HDFS-941-6.patch
          55 kB
          bc Wong

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-1001 DataXceiver and BlockReader disagree on when to send/recv CHECKSUM_OK

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3359 DFSClient.close should close cached sockets

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1599 Umbrella Jira for Improving HBASE support in HDFS

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-3529 Add search to HBase

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-2071 Use of isConnected() in DataXceiver is invalid

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-380 support for persistent connections to improve random read performance.

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1148 Convert FSDataset to ReadWriteLock

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              bcwalrus bc Wong
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: