Problem in Kaha persistence with file name lengths and collisions during persistence

• Key: AMQ-1663
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Blocker
• Assignee: Rob Davies
• Reporter: Kevin Yaussy
• Component/s: Broker
• Affects Version/s: 5.1.0
• Fix Version/s: 5.1.0

Description

The management of file names within Kaha persistence is such that it may create a file with a name that ends up colliding with a file meant for a different, but very close, topic name. The file names are ultimately managed by org/apache/activemq/util/IOHelper.java, which will trim the file name down to a maximum file name size, default of 64. With sufficiently large topic names, it is possible to create a collision with this scheme. Ours is large enough, that even if I push the max to 200 (attempting to stay under the 256 max length in Solaris), I can still get a collision.

What this results in is modifications to a file for two different items and/or consumers, which causes corruption and errors.

Comments

[Rob Davies]

You can set the following system properties:

MaximumDirNameLength
MaximumFileNameLength

to configure your way around this

[Kevin Yaussy]

Rob,

This is not a viable option. As I indicated, I already did that - I set it to 200. But, fundamentally, given a sufficiently large topic naming convention, it is possible to create a collision.

In our environment, we actually do not need to recover an AMQBroker on existing files, so our AMQBroker restart just deletes the persistent directory. However, we do utilize and need the persistence. So, to get around this problem right now, I've changed the HashIndex.java to use a static sequence number to name the files, rather than using the whole name. I started out using name.hashCode(), which I think would be fine, but could in theory have a collision. Seems to me the final solution would need to incorporate some sort of file index file, so that you could map the "real" names to some opaque naming convention that would guarantee the file name is not too long, and also guarantee no file name collisions.