[NUTCH-2989] Can't have username/pw AND https on elastic-indexer?!

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.19
• Fix Version/s: 1.20
• Component/s: indexer, plugin
• Assignee: Tim Allison
• Reporter: Tim Allison
• Created: 01/Mar/23 20:28
• Updated: 13/Mar/24 14:51
• Resolved: 28/Aug/23 14:30

Description

While working on NUTCH-2920, I copied+pasted the elastic indexer. As part of that process, I noticed that basic auth doesn't work with https.

if (auth) {
        restClientBuilder
            .setHttpClientConfigCallback(new HttpClientConfigCallback() {
              @Override
              public HttpAsyncClientBuilder customizeHttpClient(
                  HttpAsyncClientBuilder arg0) {
                return arg0.setDefaultCredentialsProvider(credentialsProvider);
              }
            });
      }

      // In case of HTTPS, set the client up for ignoring problems with self-signed
      // certificates and stuff
      if ("https".equals(scheme)) {
        try {
          SSLContextBuilder sslBuilder = SSLContexts.custom();
          sslBuilder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
          final SSLContext sslContext = sslBuilder.build();

          restClientBuilder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
              // ignore issues with self-signed certificates
              httpClientBuilder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
              return httpClientBuilder.setSSLContext(sslContext);
            }
          });
        } catch (Exception e) {
          LOG.error("Error setting up SSLContext because: " + e.getMessage(), e);
        }
      }

On NUTCH-2920, I fixed this for the opensearch-indexer by adding another if (auth) statement under the https branch.

If this is an actual issue, I'm happy to open a PR. If I've misunderstood the code or the design, please close as "not a problem".

Issue Links

• links to: GitHub Pull Request #768