Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-387

LocalJobRunner assigns duplicate mapid's

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.4.0
    • 0.5.0
    • None
    • None
    • xp, cygwin, jdk1.4.2_12

    Description

      While hunting down nutch issue NUTCH-266 i discovered that id's are generated with following fragment of code:

      private String newId()

      { return Integer.toString(Math.abs(new Random().nextInt()),36); }

      and the related Javadoc:
      "
      public Random()

      Creates a new random number generator. Its seed is initialized to a value based on the current time:

      public Random()

      { this(System.currentTimeMillis()); }

      Two Random objects created within the same millisecond will have the same sequence of random numbers.

      "

      it appears that in this case there are more than one Random pobject generated at the same millisecond and id's are
      no longer unique.

      Attachments

        1. hadoop-fix-dup-id-trunk.patch
          0.7 kB
          Sami Siren
        2. hadoop-fix-dup-id-0.4.patch
          0.7 kB
          Sami Siren

        Activity

          People

            Unassigned Unassigned
            siren Sami Siren
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: