Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This is a maven build issue.

      Jersey 1.8 is pulling in version 1.7.1 of Jackson. Meanwhile, we are manually specifying that we want version 1.8 of Jackson in the POM files. This causes a conflict where Jackson produces unexpected results when serializing Map objects.

      How to reproduce: try this code:

      ObjectMapper mapper = new ObjectMapper();
      Map<String, Object> m = new HashMap<String, Object>();
      mapper.writeValue(new File("foo"), m);

      You will get an exception:

      Exception in thread "main" java.lang.NoSuchMethodError: org.codehaus.jackson.type.JavaType.isMapLikeType()Z
      at org.codehaus.jackson.map.ser.BasicSerializerFactory.buildContainerSerializer(BasicSerializerFactory.java:396)
      at org.codehaus.jackson.map.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:267)

      Basically the inconsistent versions of various Jackson components are causing this NoSuchMethod error.

      As far as I know, this only occurs when serializing maps-- that's why it hasn't been found and fixed yet.

      1. HADOOP-8104.patch
        2 kB
        Colin Patrick McCabe
      2. HADOOP-8104.patch
        0.9 kB
        Alejandro Abdelnur
      3. HADOOP-7470.patch
        0.9 kB
        Alejandro Abdelnur
      4. dependency-tree-old.txt
        315 kB
        Colin Patrick McCabe

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #245 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/245/)
        svn merge -c 1294784 FIXES: HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1332802)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1332802
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #245 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/245/ ) svn merge -c 1294784 FIXES: HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1332802) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1332802 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Robert Joseph Evans added a comment -

        I just pulled this into 0.23.3

        Show
        Robert Joseph Evans added a comment - I just pulled this into 0.23.3
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1005 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1005/)
        HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1294784)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1005 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1005/ ) HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1294784) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #211 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/211/)
        Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #211 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/211/ ) Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #183 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/183/)
        Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #183 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/183/ ) Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #970 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/970/)
        HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1294784)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #970 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/970/ ) HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1294784) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1806 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1806/)
        HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1294784)

        Result = ABORTED
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1806 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1806/ ) HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1294784) Result = ABORTED tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #610 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/610/)
        Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788)

        Result = ABORTED
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #610 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/610/ ) Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788) Result = ABORTED tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1798 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1798/)
        HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1294784)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1798 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1798/ ) HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1294784) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #598 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/598/)
        Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #598 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/598/ ) Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #610 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/610/)
        Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #610 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/610/ ) Merge -r 1294783:1294784 from trunk to branch. FIXES: HADOOP-8104 (Revision 1294788) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294788 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1872 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1872/)
        HADOOP-8104. Inconsistent Jackson versions (tucu) (Revision 1294784)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1872 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1872/ ) HADOOP-8104 . Inconsistent Jackson versions (tucu) (Revision 1294784) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294784 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Alejandro Abdelnur added a comment -

        committed to trunk and branch-0.23

        Show
        Alejandro Abdelnur added a comment - committed to trunk and branch-0.23
        Hide
        Alejandro Abdelnur added a comment -

        re-uploading patch this time with the correct name.

        Show
        Alejandro Abdelnur added a comment - re-uploading patch this time with the correct name.
        Hide
        Eli Collins added a comment -

        Oops, wrong patch name, I meant HADOOP-7470.patch (got confused that the jira # doesn't match the patch name, let's upload a new patch named HADOOP-8104 to minimize confusion)

        Show
        Eli Collins added a comment - Oops, wrong patch name, I meant HADOOP-7470 .patch (got confused that the jira # doesn't match the patch name, let's upload a new patch named HADOOP-8104 to minimize confusion)
        Hide
        Eli Collins added a comment -

        +1 to HADOOP-8104.patch

        Show
        Eli Collins added a comment - +1 to HADOOP-8104 .patch
        Hide
        Colin Patrick McCabe added a comment -

        Hi Enis,

        I tested both patches, and they work. See my comments above.

        Show
        Colin Patrick McCabe added a comment - Hi Enis, I tested both patches, and they work. See my comments above.
        Hide
        Enis Soztutar added a comment -

        @Alejandro,
        Thanks for explaining. I did not notice that the changes are in dependency management rather than explicit deps. I think the main issue here was caused by the fact that jackson-mapper is a direct dependency, but other jackson dependencies are transitive deps through jersey, and maven did not bring up 1.8.8 jackson-core-asl from jackson-mapper 1.8.8, but reused 1.7.1.

        [INFO] +- org.apache.avro:avro:jar:1.5.3:compile
        [INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
        [INFO] |  |  \- org.codehaus.jackson:jackson-core-asl:jar:1.7.1:compile
        

        Anyway, I think the patch solves the issue. Colin, have you tested it?

        Show
        Enis Soztutar added a comment - @Alejandro, Thanks for explaining. I did not notice that the changes are in dependency management rather than explicit deps. I think the main issue here was caused by the fact that jackson-mapper is a direct dependency, but other jackson dependencies are transitive deps through jersey, and maven did not bring up 1.8.8 jackson-core-asl from jackson-mapper 1.8.8, but reused 1.7.1. [INFO] +- org.apache.avro:avro:jar:1.5.3:compile [INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile [INFO] | | \- org.codehaus.jackson:jackson-core-asl:jar:1.7.1:compile Anyway, I think the patch solves the issue. Colin, have you tested it?
        Hide
        Alejandro Abdelnur added a comment -

        @Enis first comment, my patch is adding those jackson artifacts to the dependency management section, this means that they are not added to Hadoop classpath unless somebody add them to the a dependencies section explicitly or they are pulled as transitive dependency. In the current case is the later. So, what the addition to the dependency management section does is to force a particular version in case somebody is bring the artifact in. Hope this clarifies.

        Show
        Alejandro Abdelnur added a comment - @Enis first comment, my patch is adding those jackson artifacts to the dependency management section, this means that they are not added to Hadoop classpath unless somebody add them to the a dependencies section explicitly or they are pulled as transitive dependency. In the current case is the later. So, what the addition to the dependency management section does is to force a particular version in case somebody is bring the artifact in. Hope this clarifies.
        Hide
        Enis Soztutar added a comment -

        You should not exclude those jackson artifacts that jersey pulls in unless you are 100% certain they are not use by jersey. If you do the exclusion, no artifact will come at all (this is for ackson-core-asl/jackson-jaxrs/jackson-xc)

        If we are going to add jackson-(core-asl|jaxrs|xc) as explicit dependencies in pom, then we should exclude them from jersey so that jersey does not bring it's own versions. I am not suggesting just excluding them, but not adding them to pom.

        Regarding about conflicting versions, Maven version resolution rules may seem complicated but they are very precise and predictable.

        Not a maven expert myself, that is what was discussed in the linked StackOverflow question. But in any case, we shouldn't create a version conflict in the first place.

        Show
        Enis Soztutar added a comment - You should not exclude those jackson artifacts that jersey pulls in unless you are 100% certain they are not use by jersey. If you do the exclusion, no artifact will come at all (this is for ackson-core-asl/jackson-jaxrs/jackson-xc) If we are going to add jackson-(core-asl|jaxrs|xc) as explicit dependencies in pom, then we should exclude them from jersey so that jersey does not bring it's own versions. I am not suggesting just excluding them, but not adding them to pom. Regarding about conflicting versions, Maven version resolution rules may seem complicated but they are very precise and predictable. Not a maven expert myself, that is what was discussed in the linked StackOverflow question. But in any case, we shouldn't create a version conflict in the first place.
        Hide
        Colin Patrick McCabe added a comment -

        I think we can go with the Alejandro's version of the patch which adds remaining jackson artifact

        Sounds good to me.

        Show
        Colin Patrick McCabe added a comment - I think we can go with the Alejandro's version of the patch which adds remaining jackson artifact Sounds good to me.
        Hide
        Alejandro Abdelnur added a comment -

        You should not exclude those jackson artifacts that jersey pulls in unless you are 100% certain they are not use by jersey. If you do the exclusion, no artifact will come at all (this is for ackson-core-asl/jackson-jaxrs/jackson-xc)

        Regarding about conflicting versions, Maven version resolution rules may seem complicated but they are very precise and predictable.

        Show
        Alejandro Abdelnur added a comment - You should not exclude those jackson artifacts that jersey pulls in unless you are 100% certain they are not use by jersey. If you do the exclusion, no artifact will come at all (this is for ackson-core-asl/jackson-jaxrs/jackson-xc) Regarding about conflicting versions, Maven version resolution rules may seem complicated but they are very precise and predictable.
        Hide
        Enis Soztutar added a comment -

        I think we can go with the Alejandro's version of the patch which adds remaining jackson artifacts, but we should also exclude jackson dependency from jersey. Maven is said to be undeterministic about conflicting versions.

        Show
        Enis Soztutar added a comment - I think we can go with the Alejandro's version of the patch which adds remaining jackson artifacts, but we should also exclude jackson dependency from jersey. Maven is said to be undeterministic about conflicting versions.
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Colin for verifying things work.

        Yes, dependencies have to be properly curated, HADOOP-7538, we have not done this yet.

        Show
        Alejandro Abdelnur added a comment - Thanks Colin for verifying things work. Yes, dependencies have to be properly curated, HADOOP-7538 , we have not done this yet.
        Hide
        Colin Patrick McCabe added a comment -

        Just a note-- I did more than just verify that the 1.7.1 jars were missing. I also checked that Jackson didn't crash, which was the original problem.

        Show
        Colin Patrick McCabe added a comment - Just a note-- I did more than just verify that the 1.7.1 jars were missing. I also checked that Jackson didn't crash, which was the original problem.
        Hide
        Colin Patrick McCabe added a comment -

        Hi Alejandro,

        I have confirmed that the patch you attached also solves the problem. With it, I only get jackson 1.8.8 .jar files in my install directory-- no 1.7.1 jars.

        I guess the unanswered question is whether Jersey 1.8 can handle Jackson 1.8.8, which is not what it was originally designed to work against.

        It seems that we both proposed patches that solve the problem. I don't have a strong preference for which fix we take. I do feel that we should add a unit test to keep this from happening again. It seems that serializing a map is sufficient to expose this particular version skew problem.

        Show
        Colin Patrick McCabe added a comment - Hi Alejandro, I have confirmed that the patch you attached also solves the problem. With it, I only get jackson 1.8.8 .jar files in my install directory-- no 1.7.1 jars. I guess the unanswered question is whether Jersey 1.8 can handle Jackson 1.8.8, which is not what it was originally designed to work against. It seems that we both proposed patches that solve the problem. I don't have a strong preference for which fix we take. I do feel that we should add a unit test to keep this from happening again. It seems that serializing a map is sufficient to expose this particular version skew problem.
        Hide
        Enis Soztutar added a comment -

        HADOOP-7470 tried to fix the exact same problem on hbase's site, but with Jackson 1.5.5. The problem is that ivy in 1.0 depends on jersey master (meaning jackson won't be pulled), while trunk maven pulls jersey's dependencies. Since we need Jackson independently of Jersey, I think we should exclude the jackson dependency from jersey. Have you tested whether jersey 1.8 works with jackson 1.8.8.

        Show
        Enis Soztutar added a comment - HADOOP-7470 tried to fix the exact same problem on hbase's site, but with Jackson 1.5.5. The problem is that ivy in 1.0 depends on jersey master (meaning jackson won't be pulled), while trunk maven pulls jersey's dependencies. Since we need Jackson independently of Jersey, I think we should exclude the jackson dependency from jersey. Have you tested whether jersey 1.8 works with jackson 1.8.8.
        Hide
        Alejandro Abdelnur added a comment -

        There is no need to bump up jersey to get the right version of jackson. What you need to do is to define explicitly in the dependencymanagement section in hadoop-project/pom.xml the version of the jackson you want for the jackson artifacts not directly used by Hadoop (but brought it by jersey).

        The attached patch adds jackson-core-asl/jackson-jaxrs/jackson-xc versions 1.8.8 to the dependencymanagement section.

        I've run a 'mvn dependency:tree' for the whole project and only jackson 1.8.8 artifacts are being resolved with this patch.

        Show
        Alejandro Abdelnur added a comment - There is no need to bump up jersey to get the right version of jackson. What you need to do is to define explicitly in the dependencymanagement section in hadoop-project/pom.xml the version of the jackson you want for the jackson artifacts not directly used by Hadoop (but brought it by jersey). The attached patch adds jackson-core-asl/jackson-jaxrs/jackson-xc versions 1.8.8 to the dependencymanagement section. I've run a 'mvn dependency:tree' for the whole project and only jackson 1.8.8 artifacts are being resolved with this patch.
        Hide
        Colin Patrick McCabe added a comment -

        It seems that the bug was introduced by this change: https://issues.apache.org/jira/browse/HADOOP-7470

        Previously to that, we were at Jackson 1.7.1 in trunk, which was the same version that Jersey was pulling in.

        Show
        Colin Patrick McCabe added a comment - It seems that the bug was introduced by this change: https://issues.apache.org/jira/browse/HADOOP-7470 Previously to that, we were at Jackson 1.7.1 in trunk, which was the same version that Jersey was pulling in.
        Hide
        Colin Patrick McCabe added a comment -

        I just checked, and it seems that HBASE already uses a different version of both Jersey and Jackson. So I don't think it will be affected by this change. It's also worth noting that they have this comment:

        code
        <!-- While jackson is also a dependency of both jersey and avro, these
        can bring in jars from different, incompatible versions. We force
        the same version with these dependencies -->
        code

        By the way, I DID try adding those dependencies to HDFS, but it didn't resolve the version problem for HDFS.

        Show
        Colin Patrick McCabe added a comment - I just checked, and it seems that HBASE already uses a different version of both Jersey and Jackson. So I don't think it will be affected by this change. It's also worth noting that they have this comment: code <!-- While jackson is also a dependency of both jersey and avro, these can bring in jars from different, incompatible versions. We force the same version with these dependencies --> code By the way, I DID try adding those dependencies to HDFS, but it didn't resolve the version problem for HDFS.
        Hide
        Colin Patrick McCabe added a comment -

        Hi Alejandro,

        I have posted the complete dependency tree in case you want to look at it. You can get the same output yourself by running mvn dependency:tree. The project that is pulling in jackson 1.7.1 appears to be hadoop-common. The reason why it is pulled in is because Jersey 1.8 explicitly depends on jackson 1.7.1, and not a later version. The mixed versions are creating runtime exceptions.

        There is a StackOverflow question about this exact problem at http://stackoverflow.com/questions/6537287/jersey-and-jackson-maven-dependency-issues

        Show
        Colin Patrick McCabe added a comment - Hi Alejandro, I have posted the complete dependency tree in case you want to look at it. You can get the same output yourself by running mvn dependency:tree . The project that is pulling in jackson 1.7.1 appears to be hadoop-common. The reason why it is pulled in is because Jersey 1.8 explicitly depends on jackson 1.7.1, and not a later version. The mixed versions are creating runtime exceptions. There is a StackOverflow question about this exact problem at http://stackoverflow.com/questions/6537287/jersey-and-jackson-maven-dependency-issues
        Hide
        Alejandro Abdelnur added a comment -

        @Colin's dependency tree. In which Hadoop maven subproject are you getting that dependency tree output? This seems odd as jackson is defined in the dependencyManagement section with version 1.8.8

        Show
        Alejandro Abdelnur added a comment - @Colin's dependency tree. In which Hadoop maven subproject are you getting that dependency tree output? This seems odd as jackson is defined in the dependencyManagement section with version 1.8.8
        Hide
        Colin Patrick McCabe added a comment -

        I wouldn't expect it to, but we definitely should do some testing to make sure. I will post the results here as soon as I get them.

        Show
        Colin Patrick McCabe added a comment - I wouldn't expect it to, but we definitely should do some testing to make sure. I will post the results here as soon as I get them.
        Hide
        Todd Lipcon added a comment -

        Will this now break HBase or other projects which also use Jersey? HBase appears to use jersey 1.4.

        Show
        Todd Lipcon added a comment - Will this now break HBase or other projects which also use Jersey? HBase appears to use jersey 1.4.
        Hide
        Colin Patrick McCabe added a comment -
        • Bump the Jersey version from 1.8 to 1.9.1
        • Standardize on Jackson 1.8.3 to avoid inconsistency.
        Show
        Colin Patrick McCabe added a comment - Bump the Jersey version from 1.8 to 1.9.1 Standardize on Jackson 1.8.3 to avoid inconsistency.
        Hide
        Colin Patrick McCabe added a comment -

        It's pulled in from jersey. I have a patch which fixes this. Will post in a sec.

        From mvn dependency:tree

        [INFO] +- com.sun.jersey:jersey-json:jar:1.8:compile   
        [INFO] |  +- org.codehaus.jettison:jettison:jar:1.1:compile
        [INFO] |  |  \- stax:stax-api:jar:1.0.1:compile
        [INFO] |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
        [INFO] |  |  \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
        [INFO] |  |     \- javax.activation:activation:jar:1.1:compile
        [INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.7.1:compile
        [INFO] |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.7.1:compile 
        [INFO] |  \- org.codehaus.jackson:jackson-xc:jar:1.7.1:compile
        
        Show
        Colin Patrick McCabe added a comment - It's pulled in from jersey. I have a patch which fixes this. Will post in a sec. From mvn dependency:tree [INFO] +- com.sun.jersey:jersey-json:jar:1.8:compile [INFO] | +- org.codehaus.jettison:jettison:jar:1.1:compile [INFO] | | \- stax:stax-api:jar:1.0.1:compile [INFO] | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile [INFO] | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile [INFO] | | \- javax.activation:activation:jar:1.1:compile [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.7.1:compile [INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.7.1:compile [INFO] | \- org.codehaus.jackson:jackson-xc:jar:1.7.1:compile
        Hide
        Alejandro Abdelnur added a comment -

        trunk POMs specify jackson 1.8.8, where are you seeing 1.7.1 being pulled?

        Show
        Alejandro Abdelnur added a comment - trunk POMs specify jackson 1.8.8, where are you seeing 1.7.1 being pulled?

          People

          • Assignee:
            Alejandro Abdelnur
            Reporter:
            Colin Patrick McCabe
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development