Pig
  1. Pig
  2. PIG-3836

Pig signature has has guava version dependency

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      We currently use Hashing.goodFastHash(32) to generate signatures for the logical plan. Under the hood, this is a murmur32 hash.

      Guava 11, which pig directly depends on, always seeds the hash with 0.

      http://docs.guava-libraries.googlecode.com/git-history/v11.0/javadoc/src-html/com/google/common/hash/Hashing.html#line.85

      In future versions of Guava, it is seeded by current time: http://docs.guava-libraries.googlecode.com/git-history/v14.0/javadoc/src-html/com/google/common/hash/Hashing.html#line.47

      So when future versions of guava is pulled into the classpath (which if often), we no longer get the same signature for the same logical plan between each executions of pig. This introduces unnecessary complexity for managing the classpath.

        Activity

        Hide
        Cheolsoo Park added a comment -

        Committed to trunk.

        Show
        Cheolsoo Park added a comment - Committed to trunk.
        Hide
        Akihiro Matsukawa added a comment -

        Thanks!

        Show
        Akihiro Matsukawa added a comment - Thanks!
        Hide
        Cheolsoo Park added a comment -

        +1. I'll commit it tomorrow.

        Thank you Akihiro!

        Show
        Cheolsoo Park added a comment - +1. I'll commit it tomorrow. Thank you Akihiro!
        Hide
        Akihiro Matsukawa added a comment -

        Patch should have said "murmur3_32" instead of "murmur_32". Fixed.

        Show
        Akihiro Matsukawa added a comment - Patch should have said "murmur3_32" instead of "murmur_32". Fixed.
        Hide
        Cheolsoo Park added a comment -

        Akihiro Matsukawa, thank you for the patch. But it breaks the compilation.

            [javac] /Users/cheolsoop/workspace/pig-apache/src/org/apache/pig/newplan/logical/relational/LogicalPlan.java:130: cannot find symbol
            [javac] symbol  : method murmur_32(int)
            [javac] location: class com.google.common.hash.Hashing
            [javac]         HashFunction hf = Hashing.murmur_32(0);
            [javac]                                  ^
        

        Did you mean murmur3_32(int)? I see it in JavaDoc here.

        Show
        Cheolsoo Park added a comment - Akihiro Matsukawa , thank you for the patch. But it breaks the compilation. [javac] /Users/cheolsoop/workspace/pig-apache/src/org/apache/pig/newplan/logical/relational/LogicalPlan.java:130: cannot find symbol [javac] symbol : method murmur_32( int ) [javac] location: class com.google.common.hash.Hashing [javac] HashFunction hf = Hashing.murmur_32(0); [javac] ^ Did you mean murmur3_32(int) ? I see it in JavaDoc here .
        Hide
        Akihiro Matsukawa added a comment -

        Attached patch directly uses a murmur hash seeded with 0 to remove this version dependency on guava.

        Show
        Akihiro Matsukawa added a comment - Attached patch directly uses a murmur hash seeded with 0 to remove this version dependency on guava.

          People

          • Assignee:
            Akihiro Matsukawa
            Reporter:
            Akihiro Matsukawa
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development