[CASSANDRA-6436] AbstractColumnFamilyInputFormat does not use start and end tokens configured via ConfigHelper.setInputRange()

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.0.7
• Component/s: Legacy/Tools
• Labels: hadoop patch
• Severity: Normal
• Since Version: 1.2.6
• Assignee: Paulo Motta
• Reporter: Paulo Motta

Authors

Paulo Motta

Reviewers

Piotr Kolaczkowski

• Created: 03/Dec/13 14:30
• Updated: 16/Apr/19 09:31
• Resolved: 26/Mar/14 15:38

Description

ConfigHelper allows to set a token input range via the setInputRange(conf, startToken, endToken) call (ConfigHelper:254).

We used this feature to limit a hadoop job range to a single Cassandra node's range, or even to single row key, mostly for testing purposes.

This worked before the fix for CASSANDRA-5536 (https://github.com/apache/cassandra/commit/aaf18bd08af50bbaae0954d78d5e6cbb684aded9), but after this ColumnFamilyInputFormat never uses the value of KeyRange.start_token when defining the input splits (AbstractColumnFamilyInputFormat:142-160), but only KeyRange.start_key, which needs an order preserving partitioner to work.

I propose the attached fix in order to allow defining Cassandra token ranges for a given Hadoop job even when using a non-order preserving partitioner.

Example use of ConfigHelper.setInputRange(conf, startToken, endToken) to limit the range to a single Cassandra Key with RandomPartitioner:

IPartitioner part = ConfigHelper.getInputPartitioner(job.getConfiguration());
Token token = part.getToken(ByteBufferUtil.bytes("Cassandra Key"));
BigInteger endToken = (BigInteger) new BigIntegerConverter().convert(BigInteger.class, part.getTokenFactory().toString(token));
BigInteger startToken = endToken.subtract(new BigInteger("1"));
ConfigHelper.setInputRange(job.getConfiguration(), startToken.toString(), endToken.toString());