Avro
  1. Avro
  2. AVRO-1037

Problems using Avro 1.6.2 with Hadoop (CDH3 or 1.0)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2
    • Fix Version/s: 1.6.3
    • Component/s: java
    • Labels:
      None

      Description

      AVRO-972 uses a method on JsonNode that was introduced with Jackson 1.6.0. Unfortunately, Hadoop in a distributed environment places jackson on the mapreduce classpath prior to a user's Jar.

      Avro should function when this happens.

      There are two things we are using that are new in Jackson 1.6.0 – JsonNode.getValueAsDouble() and MinimalPrettyPrinter

      MinimalPrettyPrinter is OK because the class is new, and if there is a Jackson 1.5.x jar on a classpath before one that is >= 1.6.0, it will find the class in the second jar. We need to avoid JsonNode.getValueAsDouble() however. There is a replacement that is equivalent and existed prior, JsonNode.getDoubleValue().

      1. AVRO-1037.patch
        0.7 kB
        Scott Carey

        Activity

        Hide
        Scott Carey added a comment -

        Committed revision 1293464

        Show
        Scott Carey added a comment - Committed revision 1293464
        Hide
        Doug Cutting added a comment -

        +1

        Show
        Doug Cutting added a comment - +1
        Hide
        Scott Carey added a comment -

        If there are no objections, I will check this in tomorrow. I would like to get a 1.6.3 with this in it out soon.

        Show
        Scott Carey added a comment - If there are no objections, I will check this in tomorrow. I would like to get a 1.6.3 with this in it out soon.
        Hide
        Scott Carey added a comment -

        The stack trace I ran into was:

        java.lang.NoSuchMethodError: org.codehaus.jackson.JsonNode.getValueAsDouble()D
                at org.apache.avro.Schema$Field.defaultValueEquals(Schema.java:461)
                at org.apache.avro.Schema$Field.equals(Schema.java:451)
                at java.util.AbstractList.equals(AbstractList.java:507)
                at org.apache.avro.Schema$RecordSchema.equals(Schema.java:647)
                at java.util.HashMap.put(HashMap.java:376)
        
        Show
        Scott Carey added a comment - The stack trace I ran into was: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonNode.getValueAsDouble()D at org.apache.avro.Schema$Field.defaultValueEquals(Schema.java:461) at org.apache.avro.Schema$Field.equals(Schema.java:451) at java.util.AbstractList.equals(AbstractList.java:507) at org.apache.avro.Schema$RecordSchema.equals(Schema.java:647) at java.util.HashMap.put(HashMap.java:376)

          People

          • Assignee:
            Scott Carey
            Reporter:
            Scott Carey
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development