Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11152

Better random number generator

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      HDFS-7122 showed that naive ThreadLocal usage of simple LCG based j.u.Random creates unacceptable distribution of random numbers for block placement. Similarly, ThreadLocalRandom in java 7 (same static thread local with synchronized methods overridden) has the same problem.

      "Better" is defined as better quality and faster than j.u.Random (which is already much faster (20x) than SecureRandom).

      People (e.g. Numerical Recipes) have shown that by combining LCG and XORShift we can have a better fast RNG. It'd be worthwhile to investigate a thread local version of these "better" RNG.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            vicaya Luke Lu

            Dates

              Created:
              Updated:

              Slack

                Issue deployment