Hadoop Common
  1. Hadoop Common
  2. HADOOP-7606

Upgrade Jackson to version 1.7.1 to match the version required by Jersey

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 0.23.0, 0.24.0
    • Component/s: test
    • Labels:
      None

      Description

      As of 2 days ago, 13 tests started failing, all with errors in Avro-related tests.

      1. HADOOP-7606v1.patch
        0.4 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          Git bisect blames commit cdfe85647d3492e0c803f1bc65901cded37f766b, which is HADOOP-7594.

          The errors are all along the lines of:

          java.lang.NoSuchMethodError: org.codehaus.jackson.type.JavaType.<init>(Ljava/lang/Class;)V
          	at org.codehaus.jackson.map.type.TypeBase.<init>(TypeBase.java:15)
          	at org.codehaus.jackson.map.type.SimpleType.<init>(SimpleType.java:45)
          	at org.codehaus.jackson.map.type.SimpleType.<init>(SimpleType.java:40)
          	at org.codehaus.jackson.map.type.TypeBindings.<clinit>(TypeBindings.java:20)
          	at org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
          	at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
          	at org.codehaus.jackson.map.ObjectMapper.<clinit>(ObjectMapper.java:179)
          	at org.apache.hadoop.conf.TestConfiguration.testDumpConfiguration(TestConfiguration.java:692)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at junit.framework.TestCase.runTest(TestCase.java:168)
          	at junit.framework.TestCase.runBare(TestCase.java:134)
          	at junit.framework.TestResult$1.protect(TestResult.java:110)
          	at junit.framework.TestResult.runProtected(TestResult.java:128)
          	at junit.framework.TestResult.run(TestResult.java:113)
          	at junit.framework.TestCase.run(TestCase.java:124)
          	at junit.framework.TestSuite.runTest(TestSuite.java:232)
          	at junit.framework.TestSuite.run(TestSuite.java:227)
          	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
          	at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
          	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
          	at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)
          	at org.apache.maven.surefire.Surefire.run(Surefire.java:104)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:290)
          	at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1017)
          
          Show
          Aaron T. Myers added a comment - Git bisect blames commit cdfe85647d3492e0c803f1bc65901cded37f766b, which is HADOOP-7594 . The errors are all along the lines of: java.lang.NoSuchMethodError: org.codehaus.jackson.type.JavaType.<init>(Ljava/lang/Class;)V at org.codehaus.jackson.map.type.TypeBase.<init>(TypeBase.java:15) at org.codehaus.jackson.map.type.SimpleType.<init>(SimpleType.java:45) at org.codehaus.jackson.map.type.SimpleType.<init>(SimpleType.java:40) at org.codehaus.jackson.map.type.TypeBindings.<clinit>(TypeBindings.java:20) at org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530) at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63) at org.codehaus.jackson.map.ObjectMapper.<clinit>(ObjectMapper.java:179) at org.apache.hadoop.conf.TestConfiguration.testDumpConfiguration(TestConfiguration.java:692) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145) at org.apache.maven.surefire.Surefire.run(Surefire.java:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:290) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1017)
          Hide
          Aaron T. Myers added a comment -

          I just tried reverting HADOOP-7594 in my local branch and doing so does indeed fix the test failures.

          Show
          Aaron T. Myers added a comment - I just tried reverting HADOOP-7594 in my local branch and doing so does indeed fix the test failures.
          Hide
          Alejandro Abdelnur added a comment -

          Those testcases are failing because of HADOOP-7594 which introduced Jersey dependency, which brings in a bunch of Jackson dependencies as transitive dependencies with version 1.7.1. Among those is jackson-core-asl.

          And, hadoop-common declares jackson-mapper-asl which uses version 1.6.9 and has a jackson-core-asl as transitive dependency.

          Because hadoop-common does not declare an explicit dependency on jackson-core-asl, and because the transitive occurrences of jackson-core-asl (from jackson-mapper-asl and jersey) are at the same level, the greater version wins.

          And it seems that from 1.6.9 to 1.7.1 there are some API changes, we don't know if Jersey is using those APIs.

          The safest bet would be to upgrade hadoop-project/pom.xml to use Jackson 1.7.1 and tweak the necessary API calls that changed and we are using.

          Maven tip of the day: 'mvn dependency:tree' helps you understand how dependencies are pulled/resolved

          Show
          Alejandro Abdelnur added a comment - Those testcases are failing because of HADOOP-7594 which introduced Jersey dependency, which brings in a bunch of Jackson dependencies as transitive dependencies with version 1.7.1. Among those is jackson-core-asl . And, hadoop-common declares jackson-mapper-asl which uses version 1.6.9 and has a jackson-core-asl as transitive dependency. Because hadoop-common does not declare an explicit dependency on jackson-core-asl, and because the transitive occurrences of jackson-core-asl (from jackson-mapper-asl and jersey) are at the same level, the greater version wins. And it seems that from 1.6.9 to 1.7.1 there are some API changes, we don't know if Jersey is using those APIs. The safest bet would be to upgrade hadoop-project/pom.xml to use Jackson 1.7.1 and tweak the necessary API calls that changed and we are using. Maven tip of the day: 'mvn dependency:tree' helps you understand how dependencies are pulled/resolved
          Hide
          Arun C Murthy added a comment -

          Thanks Tucu!

          Show
          Arun C Murthy added a comment - Thanks Tucu!
          Hide
          Alejandro Abdelnur added a comment -

          for the tip?

          Show
          Alejandro Abdelnur added a comment - for the tip?
          Hide
          Alejandro Abdelnur added a comment -

          Bumping up the version of jackson-mapper to 1.7.1 in hadoop-project/pom.xml fixes the test failures.

          (More info on the failure: it seems some API from jackson-core-asl used from jackson-mapper-asl has changed, but no their public API, or at least the one we use from Hadoop)

          Show
          Alejandro Abdelnur added a comment - Bumping up the version of jackson-mapper to 1.7.1 in hadoop-project/pom.xml fixes the test failures. (More info on the failure: it seems some API from jackson-core-asl used from jackson-mapper-asl has changed, but no their public API, or at least the one we use from Hadoop)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12492855/HADOOP-7606v1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/128//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12492855/HADOOP-7606v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/128//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          +1, the patch looks good to me. I'm currently running all the common tests now. Will commit this if those pass cleanly.

          Show
          Aaron T. Myers added a comment - +1, the patch looks good to me. I'm currently running all the common tests now. Will commit this if those pass cleanly.
          Hide
          Aaron T. Myers added a comment -

          I just committed this to trunk.

          Thanks a lot for the quick turn-around, Alejandro!

          Show
          Aaron T. Myers added a comment - I just committed this to trunk. Thanks a lot for the quick turn-around, Alejandro!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #826 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/826/)
          HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #826 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/826/ ) HADOOP-7606 . Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #903 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/903/)
          HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #903 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/903/ ) HADOOP-7606 . Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #836 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/836/)
          HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #836 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/836/ ) HADOOP-7606 . Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #781 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/781/)
          HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #781 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/781/ ) HADOOP-7606 . Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #805 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/805/)
          HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #805 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/805/ ) HADOOP-7606 . Upgrade Jackson to version 1.7.1 to match the version required by Jersey (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164779 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Robert Joseph Evans added a comment -

          Can someone please commit this to branch-0.23 too?

          Show
          Robert Joseph Evans added a comment - Can someone please commit this to branch-0.23 too?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          After getting a late distress call from Karam Singh (don't worry, it is just Hadoop testing related ), I just merged this into branch 0.23 too.

          Show
          Vinod Kumar Vavilapalli added a comment - After getting a late distress call from Karam Singh (don't worry, it is just Hadoop testing related ), I just merged this into branch 0.23 too.

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Aaron T. Myers
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development