[NIFI-9641] Allow ZooKeeper to respect the chroot suffix for ZK connection strings

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.16.0
• Assignee: Nathan Gough
• Reporter: Nathan Gough
• Created: 28/Jan/22 15:55
• Updated: 01/Mar/22 11:51
• Resolved: 01/Mar/22 11:51

Description

As per a question from the mail list:

... use of the PutSolrContentStream processor had broken.  Looking through the processor code there was an upgrade to the SolrJ client and a commit in March 2020 (and referenced below) that appears to prevent nested zk chroot paths for SolrCloud connections (i.e. the zookeeper connection string is truncated).

SolrUtils.java (nifi/SolrUtils.java at master · apache/nifi · GitHub)

The commit of intrest regarding the new process for initiating a CloudSolrClient in SolrJ 

https://github.com/apache/nifi/commit/9b4292024be6fae188cb1efa3a07dc9489e9a5b4#diff-13320e5b198f236cea296fb01cb7376755d65c444678e781fa0940c2a28db88b

For a nested Solr path "/solr/PROD", "/solr/DEV", "/solr/DR" … the string is truncated to the base path only i.e. “/solr” (this is only an issue for nested chroots)

The code of interest is here in the SolrUtils.java class:

 

if (SOLR_TYPE_STANDARD.getValue().equals(context.getProperty(SOLR_TYPE).getValue())) {
            return new HttpSolrClient(solrLocation, httpClient);
            return new HttpSolrClient.Builder(solrLocation).withHttpClient(httpClient).build();
        } else {
            // CloudSolrClient.Builder now requires a List of ZK addresses and znode for solr as separate parameters
            final String zk[] = solrLocation.split("/");
            final List zkList = Arrays.asList(zk[0].split(","));
            String zkRoot = "/";
            if (zk.length > 1 && ! zk[1].isEmpty()) {
                zkRoot += zk[1];
            }

 

  

I think the issue can be resolved by changing this line of code which should capture the entire nested path and not just the base path at position zk[1] in the String array.

final String zk[] = solrLocation.split("/");

To

final String[] zk = solrLocation.split("/",2)

This page documents the ability for connection strings to include a 'chroot suffix' which executes commands relative to the ZK database path specified:
https://zookeeper.apache.org/doc/r3.6.1/apidocs/zookeeper-server/org/apache/zookeeper/ZooKeeper.html

 

Ensure that NiFi can use the suffix correctly, in particular for Solr processors. The SolrUtils.java:284 class ignores the possibility of this suffix so will need to be fixed.

Issue Links

• links to: GitHub Pull Request #5727