Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5088

HashValuePartitioner has skew when there is only map fields

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.17.0, 0.16.1
    • None
    • None
    • Reviewed

    Description

      One user who just did union of map fields, got a skew with everything going to one input file as hashcode for map was calculated on the size. knoguchi pointed out that my assumption of map hashcode being non-deterministic and can change with jdk versions is wrong as it just does a sum of hashcode of its entries.

      Attachments

        1. PIG-5088-1.patch
          2 kB
          Rohini Palaniswamy

        Activity

          People

            rohini Rohini Palaniswamy
            rohini Rohini Palaniswamy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: