[CASSANDRA-6894] During a rolling upgrade from 1.2.15 to 2.0.6 nodes throws a MarshalException while attempting to merge schema.

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate
• Severity: Normal
• Assignee: Unassigned
• Reporter: Federico Piccinini
• Created: 20/Mar/14 11:25
• Updated: 16/Apr/19 09:31
• Resolved: 20/Mar/14 13:18

Description

During a rolling upgrade from 1.2.15 to 2.0.5 nodes running 1.2.15 throw a MarshalException:

ERROR [InternalResponseStage:28] 2014-03-17 13:18:28,505 CassandraDaemon.java (line 191) Exception in thread Thread[InternalResponseStage:28,5,main]
org.apache.cassandra.db.marshal.MarshalException: unable to make long from 'column1'
        at org.apache.cassandra.db.marshal.LongType.fromString(LongType.java:87)
        at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:231)
        at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1524)
        at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1456)
        at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:306)
        at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:481)
        at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:356)
        at org.apache.cassandra.service.MigrationTask$1.response(MigrationTask.java:66)
        at org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:47)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NumberFormatException: For input string: "column1"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Long.parseLong(Long.java:441)
        at java.lang.Long.parseLong(Long.java:483)
        at org.apache.cassandra.db.marshal.LongType.fromString(LongType.java:83)
        ... 12 more

In this context we have a multi-DC, nocql setup spanning 3 DCs. No columns are explicitly defined. After the first node has been migrated to 2.0.6 and restarted, all other nodes running 1.2.15 have thrown at the same time.
We also had to restart a 1.2.15 node during the upgrade (all other nodes up and running stably, partially running 2.0.6, partially running 1.2.15). The node we had to restart refused to boot up until it was upgraded to 2.0.6 throwing the following (seemingly related) exception:

ERROR [main] 2014-03-17 13:41:30,211 CassandraDaemon.java (line 464) Exception encountered during startup
org.apache.cassandra.db.marshal.MarshalException: unable to make long from 'column1'
	at org.apache.cassandra.db.marshal.LongType.fromString(LongType.java:87)
	at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:231)
	at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1524)
	at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1456)
	at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:306)
	at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:287)
	at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:154)
	at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:588)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:253)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Caused by: java.lang.NumberFormatException: For input string: "column1"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:441)
	at java.lang.Long.parseLong(Long.java:483)
	at org.apache.cassandra.db.marshal.LongType.fromString(LongType.java:83)
	... 10 more

Issue Links

• duplicates: CASSANDRA-6700 Use real node messaging versions for schema exchange decisions