Hadoop Common
  1. Hadoop Common
  2. HADOOP-7153

MapWritable violates contract of Map interface for equals() and hashCode()

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.23.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      MapWritable now implements equals() and hashCode() based on the map contents rather than object identity in order to correctly implement the Map interface.

      Description

      o.a.h.io.MapWritable implements the java.util.Map interface, however it does not define an implementation of the equals() or hashCode() methods; instead the default implementations in java.lang.Object are used.

      This violates the contract of the Map interface which defines different behaviour for equals() and hashCode() than Object does. More information here: http://download.oracle.com/javase/6/docs/api/java/util/Map.html#equals(java.lang.Object)

      The practical consequence is that MapWritables containing equal entries cannot be compared properly. We were bitten by this when trying to write an MRUnit test for a Mapper that outputs MapWritables; the MRUnit driver cannot test the equality of the expected and actual MapWritable objects.

      1. 7153-mapwritable-equality.combined.diff
        2 kB
        Nicholas Telford
      2. 7153-mapwritable-equality-test.diff
        1.0 kB
        Nicholas Telford
      3. 7153-mapwritable-equality.diff
        0.9 kB
        Nicholas Telford

        Issue Links

          Activity

          Hide
          Nicholas Telford added a comment -

          Adds new implementations of the equals() and hashCode() methods. The hashCode() method adds 1 to the value of instance.hashCode() simply so that this.hashCode() != this.instance.hashCode().

          Any code that relies on the strict object equality behaviour from Object#equals() or Object#hashCode() within MapWritable will be affected.

          Show
          Nicholas Telford added a comment - Adds new implementations of the equals() and hashCode() methods. The hashCode() method adds 1 to the value of instance.hashCode() simply so that this.hashCode() != this.instance.hashCode(). Any code that relies on the strict object equality behaviour from Object#equals() or Object#hashCode() within MapWritable will be affected.
          Hide
          Nicholas Telford added a comment -

          New implementations of MapWritable#equals() and MapWritable#hashCode()

          Show
          Nicholas Telford added a comment - New implementations of MapWritable#equals() and MapWritable#hashCode()
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12471729/7153-mapwritable-equality-test.diff
          against trunk revision 1071364.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.io.TestMapWritable

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12471729/7153-mapwritable-equality-test.diff against trunk revision 1071364. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.io.TestMapWritable +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/294//console This message is automatically generated.
          Hide
          Nicholas Telford added a comment -

          The tests fail in the QA because I attached the changes and the tests as separate patches. Should I attach a single patch containing both to satisfy the QA bot?

          Show
          Nicholas Telford added a comment - The tests fail in the QA because I attached the changes and the tests as separate patches. Should I attach a single patch containing both to satisfy the QA bot?
          Hide
          Aaron T. Myers added a comment -

          Yes indeed. Please attach a single patch containing both the functional change and the associated test.

          Show
          Aaron T. Myers added a comment - Yes indeed. Please attach a single patch containing both the functional change and the associated test.
          Hide
          Nicholas Telford added a comment -

          Combined patch. No changes.

          Show
          Nicholas Telford added a comment - Combined patch. No changes.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12471822/7153-mapwritable-equality.combined.diff
          against trunk revision 1074005.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12471822/7153-mapwritable-equality.combined.diff against trunk revision 1074005. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/297//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          +1, looks good to me.

          Show
          Todd Lipcon added a comment - +1, looks good to me.
          Hide
          Todd Lipcon added a comment -

          Committed to trunk, thanks Nicholas.

          Show
          Todd Lipcon added a comment - Committed to trunk, thanks Nicholas.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #511 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/511/)
          HADOOP-7153. MapWritable violates contract of Map interface for equals() and hashCode(). Contributed by Nicholas Telford.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #511 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/511/ ) HADOOP-7153 . MapWritable violates contract of Map interface for equals() and hashCode(). Contributed by Nicholas Telford.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #616 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/616/)
          HADOOP-7153. MapWritable violates contract of Map interface for equals() and hashCode(). Contributed by Nicholas Telford.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #616 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/616/ ) HADOOP-7153 . MapWritable violates contract of Map interface for equals() and hashCode(). Contributed by Nicholas Telford.

            People

            • Assignee:
              Nicholas Telford
              Reporter:
              Nicholas Telford
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development