Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-941

Datanode xceiver protocol should allow reuse of a connection

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Right now each connection into the datanode xceiver only processes one operation.

      In the case that an operation leaves the stream in a well-defined state (eg a client reads to the end of a block successfully) the same connection could be reused for a second operation. This should improve random read performance significantly.

      Attachments

        1. HDFS-941-1.patch
          42 kB
          bc Wong
        2. HDFS-941-2.patch
          62 kB
          bc Wong
        3. HDFS-941-3.patch
          63 kB
          bc Wong
        4. HDFS-941-3.patch
          63 kB
          bc Wong
        5. HDFS-941-4.patch
          64 kB
          bc Wong
        6. HDFS-941-5.patch
          82 kB
          bc Wong
        7. hdfs941-1.png
          7 kB
          Kihwal Lee
        8. HDFS-941-6.patch
          55 kB
          bc Wong
        9. HDFS-941-6.patch
          55 kB
          bc Wong
        10. HDFS-941-6.patch
          55 kB
          bc Wong
        11. hdfs-941.txt
          56 kB
          Todd Lipcon
        12. hdfs-941.txt
          58 kB
          Todd Lipcon
        13. HDFS-941-6.22.patch
          55 kB
          Michael Stack
        14. hdfs-941.txt
          60 kB
          Todd Lipcon
        15. fix-close-delta.txt
          4 kB
          Todd Lipcon
        16. hdfs-941.txt
          60 kB
          Todd Lipcon
        17. 941.22.txt
          63 kB
          Michael Stack
        18. 941.22.txt
          63 kB
          Michael Stack
        19. 941.22.v2.txt
          63 kB
          Michael Stack
        20. 941.22.v3.txt
          61 kB
          Michael Stack

        Issue Links

        is blocked by

        Bug - A problem which impairs or prevents the functions of the product. HDFS-1001 DataXceiver and BlockReader disagree on when to send/recv CHECKSUM_OK

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-3359 DFSClient.close should close cached sockets

        • Critical - Crashes, loss of data, severe memory leak.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-1599 Umbrella Jira for Improving HBASE support in HDFS

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-3529 Add search to HBase

        • Major - Major loss of function.
        • Closed
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-2071 Use of isConnected() in DataXceiver is invalid

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-380 support for persistent connections to improve random read performance.

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-1148 Convert FSDataset to ReadWriteLock

        • Major - Major loss of function.
        • Patch Available

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bcwalrus bc Wong
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            38 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment