Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-14483

Bootstrap stream fails with Configuration exception merging remote schema

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • 3.11.x
    • Local/Config
    • None
    • Normal

    Description

      I configured yaml file for a seed node, and start it up, cqlsh into it, and

      create keyspace kong with replication = {'class':'SimpleStrategy','replication_factor':2};

      create user kong with password 'xxx';

      And create tables in keyspace kong.

       

      Then, in another Cassandra node, point to the seed, and start Cassandra service in the new node
      Run command "nodetool status kong" shows the new node Owns ?, seed owns 100%.
      Run command "nodetool bootstrap resume",
      Resuming bootstrap
          [2018-05-31 04:15:57,807] prepare with IP_Seed complete (progress: 0%)
          [2018-05-31 04:15:57,921] received file system_auth/roles (progress: 50%)
          [2018-05-31 04:15:57,960] session with IP_Seed complete (progress: 50%)
          [2018-05-31 04:15:57,965] Stream failed
          [2018-05-31 04:15:57,966] Error during bootstrap: Stream failed
          [2018-05-31 04:15:57,966] Resume bootstrap complete

      At the end of /var/log/cassandra/cassandra.log, there are errors:

      ERROR [InternalResponseStage:2] 2018-05-31 00:02:30,559 MigrationTask.java:95 - Configuration exception merging remote schema
      org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found cce68250-63d6-11e8-b887-09f7d93c2253; expected 41679dd0-2804-11e8-a8d4-cd6631f48e81)
              at org.apache.cassandra.config.CFMetaData.validateCompatibility(CFMetaData.java:941) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:895) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.config.Schema.updateTable(Schema.java:687) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.schema.SchemaKeyspace.updateKeyspace(SchemaKeyspace.java:1464) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1420) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1389) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1366) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.service.MigrationTask$1.response(MigrationTask.java:91) ~[apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53) [apache-cassandra-3.11.2.jar:3.11.2]
              at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) [apache-cassandra-3.11.2.jar:3.11.2]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_161]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_161]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
              at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) [apache-cassandra-3.11.2.jar:3.11.2]
              at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]

          ERROR [main] 2018-05-31 00:02:58,417 StorageService.java:1524 - Error while waiting on bootstrap to complete. Bootstrap will have to be restarted.
          java.util.concurrent.ExecutionException: org.apache.cassandra.streaming.StreamException: Stream failed
                  at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[guava-18.0.jar:na]
                  at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1519) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:977) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.StorageService.initServer(StorageService.java:682) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.StorageService.initServer(StorageService.java:613) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:379) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) [apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) [apache-cassandra-3.11.2.jar:3.11.2]
          Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
                  at org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:88) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) ~[guava-18.0.jar:na]
                  at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202) ~[guava-18.0.jar:na]
                  at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:215) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:191) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:481) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:571) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:324) ~[apache-cassandra-3.11.2.jar:3.11.2]
                  at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
          WARN  [main] 2018-05-31 00:02:58,426 StorageService.java:1013 - Some data streaming failed. Use nodetool to check bootstrap state and resume. For more, see `nodetool help bootstrap`. IN_PROGRESS

      Attachments

        Activity

          People

            Unassigned Unassigned
            Henryc Yongxin Cen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: