Depends on subtask1: This multiplexer enables takers of the FairCallQueue to withdraw in a weighted round-robin fashion to combat starvation of low-priority Schedulables.

Weights are configurable as follows:
ipc.8020.wrr-multiplexer.weights = 30, 20, 5

Means queue0 will be drawn from thirty times, queue1 twenty times, queue2 five times, and repeat.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12625146/subtask2_add_mux.patch
against trunk revision .

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3466//console

This message is automatically generated.

What if when queue0 didn't have any calls in queue and requestsLeft still > 0, while other queues have calls waiting in the queue?
I think when this happens, it'll be better moveToNextQueue immediately.

Hi yunjiong zhao, thanks for taking a look.

This is handled in the FairCallQueue: it treats the index from the Multiplexer as a suggestion and works on the first non-empty queue, starting from this suggestion. So if the Mux suggests queue 2 but queues 2, 3, and 4 are empty, the FairCallQueue will use queue 5, or whatever queue is not empty.

This way, the Mux doesn't need to know anything about what queues are.

Adding a preview of the new concurrent version of the multiplexer.

Upload latest version of patch for reference

Hi Chris, are there any prerequisites for this Jira? I guess I am not clear on how this class is intended to be used. Will callers hardcode usage of WeightedRoundRobinMultiplexer?

Also could you please add Javadocs for the public methods?

Hi Arapit, sorry for the lack of context; the mux is a small component of the FairCallQueue here: https://issues.apache.org/jira/browse/HADOOP-10282

It could even exist as a nested class, but it might be useful to make swappable to enable different behaviors

Thanks, I got that part.

However I am not sure if this class is intended to be replaceable with alternative mux implementations, or is WeightedRoundRobinMultiplexer the only option. I didn't look through HADOOP-10282 patch yet, but perhaps if there was a short description of how the classes fit together it would be easier to review.

I'll put a writeup in HADOOP-10282 and let you know when it's done

Added comments to and linked all 3 parts of the FairCallQueue. They are intended to be pluggable but I haven't gotten around to that yet since we're still experimenting.

Hi Chris, thanks for the additional clarification.

This change looks fine to me. Just a couple of nitpicks:

1. Coding style, spaces around = and operators.
2. advanceIndex and getCurrentIndex look like they can be private.

Thanks for the feedback Arpit. Uploaded new patch with the changes

+1 pending Jenkins.

Thanks for the updated patch Chris.

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12651270/HADOOP-10279.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4122//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4122//console

This message is automatically generated.

I committed this to trunk and branch-2. Thanks for the contribution Chris Li!

SUCCESS: Integrated in Hadoop-trunk-Commit #5742 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5742/)
HADOOP-10279. Create multiplexer, a requirement for the fair queue. (Contributed by Chris Li) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604090)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedRoundRobinMultiplexer.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedRoundRobinMultiplexer.java

SUCCESS: Integrated in Hadoop-Yarn-trunk #589 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/589/)
HADOOP-10279. Create multiplexer, a requirement for the fair queue. (Contributed by Chris Li) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604090)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedRoundRobinMultiplexer.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedRoundRobinMultiplexer.java

FAILURE: Integrated in Hadoop-Hdfs-trunk #1780 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1780/)
HADOOP-10279. Create multiplexer, a requirement for the fair queue. (Contributed by Chris Li) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604090)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedRoundRobinMultiplexer.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedRoundRobinMultiplexer.java

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1807 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1807/)
HADOOP-10279. Create multiplexer, a requirement for the fair queue. (Contributed by Chris Li) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604090)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedRoundRobinMultiplexer.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedRoundRobinMultiplexer.java