Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10391

sstableloader fails with client SSL enabled with non-standard keystore/truststore location

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Fix Version/s: 2.2.7, 3.0.7, 3.7
    • Component/s: Legacy/Tools
    • Labels:
      None
    • Environment:
    • Severity:
      Normal

      Description

      If client SSL is enabled, sstableloader is unable to access the keystore and truststore if they are not in the expected locations. I reproduce this issue providing -f /path/to/cassandra.yaml as well as manually using the -ks flag with the proper path to the keystore.

      For example:

      client_encryption_options:
          enabled: true
          keystore: /var/tmp/.keystore
      
      # sstableloader -d 172.31.2.240,172.31.2.241 -f /etc/dse/cassandra/cassandra.yaml Keyspace1/Standard1/
      Could not retrieve endpoint ranges:
      java.io.FileNotFoundException: /usr/share/dse/conf/.keystore
      Run with --debug to get full stack trace or --help to get help.
      #
      # sstableloader -d 172.31.2.240,172.31.2.241 -ks /var/tmp/.keystore Keyspace1/Standard1/
      Could not retrieve endpoint ranges:
      java.io.FileNotFoundException: /usr/share/dse/conf/.keystore
      Run with --debug to get full stack trace or --help to get help.
      #
      

      The full stack is:

      # sstableloader -d 172.31.2.240,172.31.2.241 -f /etc/dse/cassandra/cassandra.yaml --debug Keyspace1/Standard1/
      Could not retrieve endpoint ranges:
      java.io.FileNotFoundException: /usr/share/dse/conf/.keystore
      java.lang.RuntimeException: Could not retrieve endpoint ranges:
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:283)
      	at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:144)
      	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:95)
      Caused by: java.io.FileNotFoundException: /usr/share/dse/conf/.keystore
      	at com.datastax.bdp.transport.client.TClientSocketFactory.getSSLSocket(TClientSocketFactory.java:128)
      	at com.datastax.bdp.transport.client.TClientSocketFactory.openSocket(TClientSocketFactory.java:114)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:186)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:120)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:111)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.createThriftClient(BulkLoader.java:302)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:254)
      	... 2 more
      root@ip-172-31-2-240:/tmp/foo#
      

      .

      If I copy the keystore to the expected location, I get the same error with the truststore.

      # sstableloader -d 172.31.2.240,172.31.2.241 -f /etc/dse/cassandra/cassandra.yaml --debug Keyspace1/Standard1/
      Could not retrieve endpoint ranges:
      java.io.FileNotFoundException: /usr/share/dse/conf/.truststore
      java.lang.RuntimeException: Could not retrieve endpoint ranges:
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:283)
      	at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:144)
      	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:95)
      Caused by: java.io.FileNotFoundException: /usr/share/dse/conf/.truststore
      	at com.datastax.bdp.transport.client.TClientSocketFactory.getSSLSocket(TClientSocketFactory.java:130)
      	at com.datastax.bdp.transport.client.TClientSocketFactory.openSocket(TClientSocketFactory.java:114)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:186)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:120)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:111)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.createThriftClient(BulkLoader.java:302)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:254)
      	... 2 more
      #
      

      If I copy the truststore, it finds them both, but then fails to open them due to what I assume is a password error, even those it's present in the cassandra.yaml.

      # sstableloader -d 172.31.2.240,172.31.2.241 -f /etc/dse/cassandra/cassandra.yaml --debug Keyspace1/Standard1/
      Could not retrieve endpoint ranges:
      java.io.IOException: Failed to open transport to: 172.31.2.240:9160
      java.lang.RuntimeException: Could not retrieve endpoint ranges:
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:283)
      	at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:144)
      	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:95)
      Caused by: java.io.IOException: Failed to open transport to: 172.31.2.240:9160
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:137)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:111)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.createThriftClient(BulkLoader.java:302)
      	at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:254)
      	... 2 more
      Caused by: org.apache.thrift.transport.TTransportException: Error creating the transport
      	at org.apache.thrift.transport.TSSLTransportFactory.createSSLContext(TSSLTransportFactory.java:201)
      	at org.apache.thrift.transport.TSSLTransportFactory.getClientSocket(TSSLTransportFactory.java:165)
      	at com.datastax.bdp.transport.client.TClientSocketFactory.getSSLSocket(TClientSocketFactory.java:136)
      	at com.datastax.bdp.transport.client.TClientSocketFactory.openSocket(TClientSocketFactory.java:114)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:186)
      	at com.datastax.bdp.transport.client.TDseClientTransportFactory.openTransport(TDseClientTransportFactory.java:120)
      	... 5 more
      Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
      	at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
      	at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
      	at java.security.KeyStore.load(KeyStore.java:1445)
      	at org.apache.thrift.transport.TSSLTransportFactory.createSSLContext(TSSLTransportFactory.java:179)
      	... 10 more
      Caused by: java.security.UnrecoverableKeyException: Password verification failed
      	at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770)
      	... 13 more
      

      If I specify the password on the command line, I get the same error.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nutbunnies Andrew Hust
                Reporter:
                jmoses Jon Moses
                Authors:
                Andrew Hust
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: