Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6713

The RPC server Listener thread is a scalability bottleneck

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • ipc
    • None
    • Reviewed

    Description

      The Hadoop RPC Server implementation has a single Listener thread that reads data from the socket and puts them into a call queue. This means that this single thread can pull RPC requests off the network only as fast as a single CPU can execute. This is a scalability bottlneck in our cluster.

      Attachments

        1. HADOOP-6713-rel20.patch
          13 kB
          Bharath Mundlapudi
        2. HADOOP-6713-rel20.3.patch
          13 kB
          Bharath Mundlapudi
        3. HADOOP-6713-rel20.2.patch
          12 kB
          Bharath Mundlapudi
        4. HADOOP-6713.patch
          10 kB
          Dmytro Molkov
        5. HADOOP-6713.2.patch
          11 kB
          Dmytro Molkov

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dms Dmytro Molkov
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            19 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment