[GIRAPH-362] Address master task id for communication for master (known issue from GIRAPH-211)

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Avery Ching
• Created: 10/Oct/12 17:54
• Updated: 29/Mar/13 23:07

Description

There is a workaround from GIRAPH-211 to handle requests a little differently due to issues communicating to the master. We should fix this to be a regular request in the future.

  public void sendWritableRequest(Integer destWorkerId,
                                  InetSocketAddress remoteServer,
                                  WritableRequest request) {
    if (clientRequestIdRequestInfoMap.isEmpty()) {
      byteCounter.resetAll();
    }
    boolean registerRequest = true;
/*if[HADOOP_NON_SECURE]
else[HADOOP_NON_SECURE]*/
    if (request.getType() == RequestType.SASL_TOKEN_MESSAGE_REQUEST) {
      registerRequest = false;
    }
/*end[HADOOP_NON_SECURE]*/

    Channel channel = getNextChannel(remoteServer);
    RequestInfo newRequestInfo = new RequestInfo(remoteServer, request);
    if (registerRequest) {
      request.setClientId(clientId);
      request.setRequestId(
        addressRequestIdGenerator.getNextRequestId(remoteServer));
      ClientRequestId clientRequestId =
        new ClientRequestId(destWorkerId, request.getRequestId());
      RequestInfo oldRequestInfo = clientRequestIdRequestInfoMap.putIfAbsent(
        clientRequestId, newRequestInfo);
      if (oldRequestInfo != null) {
        throw new IllegalStateException("sendWritableRequest: Impossible to " +
          "have a previous request id = " + request.getRequestId() + ", " +
          "request info of " + oldRequestInfo);
      }
    }
    ChannelFuture writeFuture = channel.write(request);
    newRequestInfo.setWriteFuture(writeFuture);

    if (limitNumberOfOpenRequests &&
        clientRequestIdRequestInfoMap.size() > maxNumberOfOpenRequests) {
      waitSomeRequests(maxNumberOfOpenRequests);
    }
  }

Issue Links

• is related to: GIRAPH-386 ClassCastException when giraph.SplitMasterWorker=false
• is related to: GIRAPH-211 Add secure authentication to Netty IPC
• relates to: GIRAPH-601 Exception when running pagerank benchmark with 6 or more workers on a pseudodistributed setup: SendVertexRequest cannot be cast to MasterRequest