Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HBASE-3393.patch
      61 kB
      Jeff Hammerbacher
    2. HBASE-3393-2.patch
      62 kB
      Jeff Hammerbacher
    3. HBASE-3393-3.patch
      78 kB
      Jeff Hammerbacher

      Issue Links

        Activity

        Hide
        Jeff Hammerbacher added a comment -

        I also fixed a bug here where the Avro server wasn't using the port passed to it at the command line.

        Show
        Jeff Hammerbacher added a comment - I also fixed a bug here where the Avro server wasn't using the port passed to it at the command line.
        Hide
        Jeff Hammerbacher added a comment -

        Tests fail with "java.lang.NoSuchFieldError: families at org.apache.hadoop.hbase.avro.TestAvroServer.testFamilyAdminAndMetadata(TestAvroServer.java:156)" and similar failures.

        Also, won't build without manually adding JBoss repository to pom.xml as directed at http://www.jboss.org/netty/downloads.html.

        That said, wanted to get the patch up in case others can figure out the test failures.

        Show
        Jeff Hammerbacher added a comment - Tests fail with "java.lang.NoSuchFieldError: families at org.apache.hadoop.hbase.avro.TestAvroServer.testFamilyAdminAndMetadata(TestAvroServer.java:156)" and similar failures. Also, won't build without manually adding JBoss repository to pom.xml as directed at http://www.jboss.org/netty/downloads.html . That said, wanted to get the patch up in case others can figure out the test failures.
        Hide
        Jeff Hammerbacher added a comment -

        Turns out Maven won't read repositories from dependent projects. I've added the JBoss repository to the HBase pom.xml so that Avro will build correctly.

        Show
        Jeff Hammerbacher added a comment - Turns out Maven won't read repositories from dependent projects. I've added the JBoss repository to the HBase pom.xml so that Avro will build correctly.
        Hide
        Lars Francke added a comment -

        The correct JBoss repository is: http://repository.jboss.org/nexus/content/groups/public-jboss/ the "public" one also mirrors central. The Netty page should be updated.

        Show
        Lars Francke added a comment - The correct JBoss repository is: http://repository.jboss.org/nexus/content/groups/public-jboss/ the "public" one also mirrors central. The Netty page should be updated.
        Hide
        stack added a comment -

        I applied the patch along w/ Lars qualification on the jboss repo and got this compiling:

        [INFO] ------------------------------------------------------------------------
        [ERROR] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Compilation failure
        /Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java:[140,26] incompatible types
        found   : org.apache.avro.generic.GenericData.Array<org.apache.avro.util.Utf8>
        required: java.util.List<java.lang.CharSequence>
        

        You get that Jeff?

        Show
        stack added a comment - I applied the patch along w/ Lars qualification on the jboss repo and got this compiling: [INFO] ------------------------------------------------------------------------ [ERROR] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Compilation failure /Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java:[140,26] incompatible types found : org.apache.avro. generic .GenericData.Array<org.apache.avro.util.Utf8> required: java.util.List<java.lang.CharSequence> You get that Jeff?
        Hide
        Jeff Hammerbacher added a comment -

        Hey Stack,

        Huh, that's a new one for me. I was getting a different error (the NoSuchFieldError quoted above). Your error is easier to fix. I need to figure out the GenericArray --> List API change that happened during Avro 1.3.3 --> 1.4.1 and get this working. Will do what I can today.

        Regards,
        Jeff

        Show
        Jeff Hammerbacher added a comment - Hey Stack, Huh, that's a new one for me. I was getting a different error (the NoSuchFieldError quoted above). Your error is easier to fix. I need to figure out the GenericArray --> List API change that happened during Avro 1.3.3 --> 1.4.1 and get this working. Will do what I can today. Regards, Jeff
        Hide
        Jeff Hammerbacher added a comment -

        This one should work. Tests pass, and I was able to use pyhbase to throw some data at a local cluster.

        Show
        Jeff Hammerbacher added a comment - This one should work. Tests pass, and I was able to use pyhbase to throw some data at a local cluster.
        Hide
        stack added a comment -

        Applied to TRUNK. Thanks for the patch Jeff (and the amendment Lars). Let me know if you want it on branch though it probably doesn't belong there.

        Show
        stack added a comment - Applied to TRUNK. Thanks for the patch Jeff (and the amendment Lars). Let me know if you want it on branch though it probably doesn't belong there.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #1700 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1700/)
        HBASE-3393 Update Avro gateway to use Avro 1.4.1 and the new server.join() method

        Show
        Hudson added a comment - Integrated in HBase-TRUNK #1700 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1700/ ) HBASE-3393 Update Avro gateway to use Avro 1.4.1 and the new server.join() method
        Hide
        Lars Francke added a comment -

        I know this is closed but just to add that Scott Carey was incorrect in saying that repositories in dependent projects aren't picked up. We need that JBoss repository simply because the Avro 1.4.1 POM is incorrect. Should we upgrade to 1.5.0 or later we can remove it again.

        Show
        Lars Francke added a comment - I know this is closed but just to add that Scott Carey was incorrect in saying that repositories in dependent projects aren't picked up. We need that JBoss repository simply because the Avro 1.4.1 POM is incorrect. Should we upgrade to 1.5.0 or later we can remove it again.
        Hide
        stack added a comment -

        This upgrade is incompatible with hadoop 0.22; if you run hbase unit tests TestHFileOutputFormat fails because of the incompatible avros (hbase has 1.4.1 and hadoop 0.22 has 1.3.2).

        Show
        stack added a comment - This upgrade is incompatible with hadoop 0.22; if you run hbase unit tests TestHFileOutputFormat fails because of the incompatible avros (hbase has 1.4.1 and hadoop 0.22 has 1.3.2).

          People

          • Assignee:
            Jeff Hammerbacher
            Reporter:
            Jeff Hammerbacher
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development