Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10452

SASL negotation should support buffer size negotiation

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: encryption
    • Labels:

      Description

      The SASL negotation for data transfer encryption implemented in Hadoop currently only supports negotiation of cipher and QoP. The buffer size is not negotiated by SASL.

      SaslOutputStream.java
      public SaslOutputStream(OutputStream outStream, SaslClient saslClient) {
          this.saslServer = null;
          this.saslClient = saslClient;
          String qop = (String) saslClient.getNegotiatedProperty(Sasl.QOP);
          this.useWrap = qop != null && !"auth".equalsIgnoreCase(qop);
          if (useWrap) {
            this.outStream = new BufferedOutputStream(outStream, 64*1024);
          } else {
            this.outStream = outStream;
          }
        }
      
      DataTransferSaslUtil.java
      public static Map<String, String> createSaslPropertiesForEncryption(
            String encryptionAlgorithm) {
          Map<String, String> saslProps = Maps.newHashMapWithExpectedSize(3);
          saslProps.put(Sasl.QOP, QualityOfProtection.PRIVACY.getSaslQop());
          saslProps.put(Sasl.SERVER_AUTH, "true");
          saslProps.put("com.sun.security.sasl.digest.cipher", encryptionAlgorithm);
          return saslProps;
        }
      

      For applications that are sensitive to buffer size, e.g., HBase, there should be a way to configure the buffer size.

      In addition, the SASL negotiation for RPC does use the negotiated buffer size, but since Hadoop never actually negotiates it, the size is the default value, 64 KB.

      SaslRpcClient.java
      public OutputStream getOutputStream(OutputStream out) throws IOException {
          if (useWrap()) {
            // the client and server negotiate a maximum buffer size that can be
            // wrapped
            String maxBuf = (String)saslClient.getNegotiatedProperty(Sasl.RAW_SEND_SIZE);
            out = new BufferedOutputStream(new WrappedOutputStream(out),
                                           Integer.parseInt(maxBuf));
          }
          return out;
        }
      

      We should make it possible to negotiate the buffer size for both data transfer and RPC.

        Attachments

          Activity

            People

            • Assignee:
              weichiu Wei-Chiu Chuang
              Reporter:
              weichiu Wei-Chiu Chuang

              Dates

              • Created:
                Updated:

                Issue deployment