Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13382

IntegrationTestBigLinkedList should use SecureRandom

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      IntegrationTestBigLinkedList currently uses java.util.Random to generate its random keys. The keys are 128 bits long, but we generate them using Random.nextBytes(). The Random implementation itself only has a 48-bit seed, so even though we have a very long key string, it doesn't have anywhere near that amount of entropy.

      This means that after a few billion rows, it's quite likely to run into a collision: filling in a 16-byte key is equivalent to four calls to rand.nextInt(). So, for 10B rows, we are cycling through 40B different 'seed' values. With a 48-bit seed, it's quite likely we'll end up using the same seed twice, after which point any future rows generated by the colliding mappers are going to be equal. This results in broken chains and a failed verification job.

      The fix is simple – we should use SecureRandom to generate the random keys, instead.

        Attachments

        1. HBASE-13382_master_v1.patch
          2 kB
          Dima Spivak

          Issue Links

            Activity

              People

              • Assignee:
                dimaspivak Dima Spivak
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: