[HBASE-2432] enhance hbase.util.Bytes.toBytes() with length limit

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 0.20.1
• Component/s: util
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 09/Apr/10 18:16
• Updated: 11/Jun/22 23:22
• Resolved: 26/Jan/14 22:06

Description

The following stack trace is seen in our hadoop log:

java.lang.IllegalArgumentException: Row > 32767
at org.apache.hadoop.hbase.KeyValue.createByteArray(KeyValue.java:437)
at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:405)
at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:374)
at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:353)
at org.apache.hadoop.hbase.client.Put.add(Put.java:137)
at org.apache.hadoop.hbase.client.Put.add(Put.java:108)
at org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:170)
at org.apache.nutch.scoring.webgraph.ScoreUpdater$ScoreUpdaterReducer.reduce(ScoreUpdater.java:127)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:563)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

Bytes.toBytes(Float.valueOf(score).toString()) may return an array longer than 32767 bytes.

We should enhance Bytes.toBytes() to include length limit:
public static byte[] toBytes(String s, int length)

String.getBytes() doesn't have length limit